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TASK  HATCHING  EXERCISE 

Task  List  for  llB 
Form  A 


1st  2nd 

Task  Task 

2^^  Category/  Category/ 

No^  Task  Name  Activity  Activity 


1.  Prepare  a  dragon  for  firing 

2.  Techniques  of  movement  in 
urban  terrain 

3.  Call  for/adjust  indirect 
fire 

4.  Select  fire  team/scout  squad 
overwatch  position 

5.  Set  headspace  and  timing  on 
a  caliber  .50  machinegun 

6.  Place  an  AN/PVS-5  (night 
vision  goggles)  into  opera¬ 
tion 

7.  Zero  AN/PVS-4  to  an  M16A1 
rifle 

8.  Select  hasty  firing  posi¬ 
tions  in  urban  terrain 

9.  Operate  radio  set  AN/PRC-77 
or  AN/PRC-25  (AN/GRC-160  or 
AN/GRC-125  assembled  for 
manpack  operations) 

10.  Collect/ report  information  - 
-SALUTE 

11.  Recognize  and  identify 
friendly  and  threat  armored 
vehicles 


3rd 

Task 

Category/ 

Activity 


B23 


1st  2nd 

Task  Task 

Category/  Category/ 
Activity  Activity 


13.  Identify  terrain  features  on 
a  map 

14.  Perform  operator  maintenance 
on  an  M16A1  rifle,  magazine, 
and  ammunition 

15.  Load,  reduce  a  stoppage,  and 
clear  an  M60  machinegun 

16.  Prepare  a  range  card  for  an 
M60  machinegun 

17.  Engage  targets  with  an  M72A2 
LAW 

18.  Engage  enemy  targets  with 
hand  grenades 

19.  Install  and  fire/recover  an 
M18A1  Claymore  mine 

20.  Move  under  direct  fire 


21.  Move  over,  through,  or 
around  obstacles  (except 
minefields) 

22.  Camouflage  yourself  and  your 
individual  equipment 

2  3 .  Put  on ,  wear ,  and  remove 

your  M17 -series  protective 
mask  with  hood 

24.  Put  on  and  wear  protective 
clothing 


Task 

No.  Task  Name 


3rd 

Task 

Category/ 

Activity 
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Task  Name 


Task 

No. 


1st  2nd  3rd 

Task  Task  Task 

Category/  Category/  Category/ 
Activity  Activity  Activity 


25.  Perform  cardiopulmonary 
resuscitation  (CPR)  on  an 
adult  using  the  one-man 
method 

26.  Put  on  a  field  or  pressure 
dressing 

27 .  Navigate  from  one  point  on 
the  ground  to  another  point 

28.  Establish  an  observation 
post 

29.  Conduct  day  and  night 
surveillance  without  the 
aid  of  electronic  devices 

30.  Administer  nerve  agent  an¬ 
tidote  to  self  (self-aid) 
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NAME 


TASK  HATCHING  EXERCISE 

Task  List  for  63B 
Form  A 


Task 

No.  Task  Name 


1st  2nd 

Task  Task 

Category/  Category/ 
Activity  Activity 


1.  Adjust  clutch  pedal  free 
travel 

2.  Maintain  assigned  toolkit 

3.  Repair  electrical  wiring 
(truck,  cargo,  1-1/4  ton, 
4x4) 

4.  Replace  air  hydraulic 
cylinder 

5.  Replace  radiator  (truck, 
cargo,  2-1/2  ton,  6x6) 

6.  Replace  service  brakes 
(truck,  utility,  1/4  ton, 
4x4) 

7.  Replace  fuel  pxunp  (truck, 
cargo,  2-1/2,  ton,  6x6) 

8.  Replace  wheel  bearings 
(truck,  cargo,  2-1/2  ton, 
6x6) 

9.  Tow  dis2dDled  vehicle  with  5 
ton  wrecker 

10.  Perform  annual  PMCS  (inspect 
and  road  test)  (truck,  car¬ 
go,  5  ton,  6x6) 

11.  Troubleshoot  brake  system 
malfunctions  (truck,  cargo, 
1-1/4  ton,  4x4) 
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3rd 

Task 

Category/ 

Activity 


1st  2nd 

Task  Task 

Category/  Category/ 
Activity  Activity 

12 .  Troubleshoot  electrical  _ 

system  (truck,  cargo,  5  ton, 

6x6) 

13 .  Troxobleshoot  fuel  system  _ 

malfunctions  (truck,  cargo, 

2-1/2  ton,  6x6) 

14 .  Troubleshoot  service  brake  _ 

malfunctions  (truck, 

utility,  1/4  ton,  4x4) 

15.  Replace  starter  _ 


16.  Administer  first  aid  to  a 
nerve  agent  casualty  (buddy- 
aid) 

17 .  Camouflage  equipment 


18.  Perform  operator  maintenance 
on  an  M16A1  rifle,  magazine, 
and  ammunition 

19.  Determine  grid  coordinates 


20.  Determine  magnetic  azimuth 
using  compass 

21.  Load,  reduce  stoppage, 
clear  M16A1  rifle 

22.  Use  challenge  and  password 


23.  Put  on  and  wear  M17  series 
protective  mask  with  hood 


Task 

No.  Task  Name 


3rd 

Task 

Category/ 

Activity 
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Task 

No.  Task  Name 


1st  2nd 

Task  Task 

Category/  Category/ 
Activity  Activity 


24.  Put  on  and  wear  protective 
clothing  in  accordance  with 
established  (MOPP)  levels 

25.  Put  on  a  field  or  pressure 
dressing 

26.  Perform  expedient  repairs 

27.  Troxibleshoot  engine  cooling 
system 

28.  Troubleshoot  engines 

29.  Troubleshoot  steering 
system 

30.  Slave  start  disabled  vehicle 
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3rd 

Task 

Category/ 

Activity 


NAME 


TASK  MATCHING  EXERCISE 

Task  List  for  71L 
Form  A 


Task 

No. 

Task  Name 

1. 

Type  a  basic  comment  to  a 
disposition  form 

2. 

Type  a  military  letter 

3. 

Type  a  joint  messageform 
(DD  Form  173/1) 

4. 

Prepare  a  requisition  for 
publications  and/or  blank 
forms — AUTODIN  (DA  Form 
4569) 

5. 

Type  military  orders 

6. 

Type  a  memorandum 

7. 

Dispatch  outgoing  dis¬ 
tribution 

8. 

Establish  functional  files 

9. 

File  documents/correspon¬ 
dence 

10. 

Type  a  second  or  subsequent 
comment  to  a  disposition 
form 

11.  . 

Type  straight  copy  material 

12. 

Assemble  correspondence 

13. 

Receipt/ transfer  classified 
material 

1st  2nd  3rd 

Task  Task  Task 

Category/  Category/  Category/ 
Activity  Activity  Activity 
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1st  2nd 

Task  Task 

Category/  Category/ 
Activity  Activity 


14.  Safeguard  "FOR  OFFICIAL  USE 
ONLY"  (FOUO)  material 

15.  Perform  operator  main¬ 
tenance  on  an  M16A1  rifle, 
magazine,  and  ammunition 

16.  Load,  reduce  a  stoppage, 
and  clear  an  M16A1  rifle 

17.  Camouflage  yourself  and 
your  individual  equipment 

18.  Practice  noise,  light,  and 
litter  discipline 

19.  Determine  the  grid  coor¬ 
dinates  of  a  point  on  a 
military  map  using  the 
military  grid  reference 
system 

20.  Determine  a  magnetic 
azimuth  using  a  compass 

21 ♦  Put  on  a  field  or  pressure 
dressing 

22.  Maintain  an  M17  series 
protective  mask 

23.  Put  on  and  wear  an  M17 
series  protective  mask  with 
hood 

24.  Put  on  and  wear  protective 
clothing  in  accordance  with 
established  mission- 
oriented  protective  posture 
(MOPP)  levels 

25.  Know  your  rights  and 
obligations  as  a  prisoner 
of  war 

26.  Administer  nerve  agent  an¬ 
tidote  to  self  (self-aid) 


Task 

No.  Task  Name 


3rd 

Task 

Category/ 

Activity 
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Task 

No.  Task  Name 


1st  2nd 

Task  Task 

Category/  Category/ 
Activity  Activity 


27.  Receive,  maintain,  and  con¬ 
trol  office  equipment 

28.  Control  expendable/non-ex- 
pendable  supplies 


3rd 

Task 

Category/ 

Activity 
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APPENDIX  C 


JOB  ACTIVITY  INSTRUMENT 


Name: 


Rank: 

MOS: 


MOS  JOB  ACTIVITY  QUESTIONNAIRE 


Listed  below  are  53  job  activities.  For  each  activity,  we  would  like  you  to  make 
the  following  four  ratings: 


1.  Is  the  activity  a  PART  OF  THE  JOB  for  E3  and  E4  soldiers  in  the  MOS  you 
are  rating?  ^ 

^  =  No  (skip  ratings  2  -  4  and  go  to  the  next  activity) 

_X.  =  Yes  (continue  with  ratings  2  -4) 


How  IMPORTANT  or  critical  is  the  activity  for  successful  performance  for  E3 
and  E4  soldiers  in  the  MOS? 

_5_  “  Extremely  high  importance  for  E3  and  E4  soldiers.  The  activity  is 
very  central  to  the  job.  (Only  the  most  critical  activities  should 
get  this  rating.) 

_4_  =  High  importance,  '^e  activity  is  central  to  the  job,  but  it  is  not 
one  of  the  most  critical  activities. 

=  Moderate  importance.  The  activity  is  part  of  the  job,  but  it  is  not 
central  to  the  job. 

_2_  =  Low  importance.  The  activity  is  part  of  the  job,  but  there  are 
many  activities  that  are  more  important. 

_L  =  Very  minpr  importance.  The  activity  is  part  of  the  job,  but  it  is 
not  at  all  important  for  successful  job  performance. 


How  FREQUENTLY  is  the  activity  performed  by  E3  and  E4  soldiers  in  the 
MOS? 

— 2_  =  Very  frequently.  The  activity  is  performed  much  more  frequently 

than  most  other  activities. 

JL  =  Frequently.  The  activity  is  performed  about  as  often  as  other 
activities. 

^  =  Occasionally  or  infrequently.  The  activity  is  performed  much  less 
frequently  than  most  other  activities. 


Cl 


4.  How  DIFFICULT  and  demanding  is  it  for  E3  and  E4  soldiers  in  the  MOS  to 
perform  this  activity  well? 

_3_  =  Very  difficult.  It  takes  considerable  skill  and  motivation  to  perform 
this  activity  well.  Few  soldiers  can  perform  this  activity. 

_2_  =  Moderately  difficult.  It  requires  some  skill  and  motivation  to 

perform  this  activity  well,  but  most  soldiers  are  eventually  able  to 
learn  the  activity. 

^  =  Routine.  The  activity  is  relatively  easy  and  requires  little  skill. 

Almost  all  soldiers  learn  the  activity  very  quickly  and  are  able  to 
perform  the  activity  with  little  difficulty. 
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Leadership/Teamwork 


1-  Work  in  a  team  -  participate  in  work  or  combat  teams 
by  helping  resolve  conflicts,  keep  the  team  moving 
toward  its  goals,  and  maintain  morale  of  other  group 
members. 

2.  Lead  a  team  -  lead  or  direct  a  team  or  unit  as  it 
tries  to  accomplish  a  prescribed  mission  or  goal. 

3.  Support/counsel  peers  -  support  and/or  counsel  peers 
when  they  have  difficulty. 

Support/counscl  subordinates  ~  counsel  subordinates 
when  they  have  difficulty. 

5.  Coach  peers  -  instruct/train  peers  on  specific  job 
tasks. 

6-  Coach  subordinates  ••  instruct/train  subordinates  on 
specific  job  tasks. 

Communication 

7.  Make  oral  reports  ftp  individuals^  ~  make  oral  reports 
to  other  individuals. 

8.  Make  oral  reports  (to  groups^  -  give  oral  briefings  to 
a  group. 

9-  Relay  oral  instructions  -  relay  oral  instructions  on  job 
tasks  or  job  procedures  to  other  individuals  or  to 
groups. 
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Difficulty 
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10  Ask  questions  --  ask  frequent  questions,  as  in  inter¬ 
viewing  and  investigating. 

11.  Record  information  ~  record  messages  via  handwriting 
or  using  a  keyboard. 

12.  Write  brief  messages  ~  write  brief  messages  (e.g., 
memos,  day-to-day  instructions,  business  letters,  etc.) 

13.  Write  longer  reports  —  compose  and  write  longer 
formal  reports  or  articles  (i.e.,  longer  than  one  or  two 
pages). 

Use  Information 

14.  Monitor/interpret  verbal  messages  -  monitor,  identify, 
and/or  interpret  verbal  messages  (oral  or  written, 
obtained  from  radio,  teletype,  computer  terminal, 
correspondences,  etc.) 

15.  Recall  verbal  information  -  recall  verbal  information. 
(That  is,  to  perform  the  required  job  tasks,  the  in¬ 
dividual  must  rely  on  memory  for  necessary  verbal 
information  that  was  communicated  more  than  a  few 
hours  ago.  However,  this  item  does  not  refer  to 
information  presented  during  formal  training  ) 

16.  Monitor/interpret  numerical  information  -  monitor, 
identify,  and/or  interpret  numerical/quantitative  infor¬ 
mation  that  is  presented  via  written  reports,  radio, 

CRT,  or  other  electronic  equipment. 
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17.  Recall  numerical  infomiatioii  --  recall  numerical/quan- 
titative  information.  (Again,  "recall"  refers  to  the  fact 
that  the  job  requires  the  use  of  memory  to  have  the 
necessary  information  to  perform  the  job  tasks. 
However,  it  is  not  memory  for  training  content  but 
memory  for  information  acquired  as  part  of  the  job.) 

18.  Monitor/Interpret  figural  inforniation  -  monitor, 
identify,  and/or  interpret  figural  information  (e.g.,  CRT 
images,  pictures,  graphs,  terrain  features). 

19-  Recall  figural  information  —  recall  figural  information 
(e.g.,  CRT  images,  picutres,  graphs,  terrain  features). 

20.  Follow  oral  directions  -  follow  oral  directions  (e.g., 
for  how  to  complete  documents,  move  from  point  A  to 
point  B,  or  schedule  repairs  on  a  variety  of  equip¬ 
ment). 

21.  Follow  written  directions  ~  follow  written 
directions/instructions  as  part  of  normal  job  duties 
(e.g.,  follow  written  orders,  SOP  protocols,  or  technical 
manuals.) 

Perceptual  Judgements 

22.  Judge  size  and  distance  -  make  judgements  of  relative 
size  and  distance,  as  in  which  of  two  objects  is 
closer/larger  or  the  distance  to  a  target. 
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23.  Judge  location  --  orient  oneself  relative  to  location 
and  direction  from  physical  objects  or  terrain  features 
and  maintain  the  proper  orientation  while  moving  from 
point  to  point. 

24.  Judge  paths  of  moving  objects  -  make  judgements 
about  the  relative  position  of  moving  objects  (e.g., 
Where  will  two  vehicles  converge?  As  the  truck  cab 
moves,  where  will  the  trailer  go?). 

Problem  Solving/Troubleshooting 

25.  Solve  electrical  system  problems  —  troubleshoot 
electrical  equipment  problems  (i.e.,  identify  the  problem 
and  suggest  solution). 

26.  Solve  mechanical  system  problems  —  troubleshoot 
mechanical  equipment  problems  (i.e.,  identify  the 
problem  and  suggest  solution). 

27.  Solve  logistical  problems  -  identify  and  solve  logistical 
problems. 

28.  Solve  tactical  manuever  problem.s  --  identify  and  solve 
tactical  maneauver  problems  (e.g.,  positioning  of  teams 
in  the  field,  countering  moves  of  enemy). 

29.  Solve  administrative  problem.s  —  identify  and  solve 
administrative  problems  (e.g.,  correct  record  keeping 
errors,  personnel  scheduling). 
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30.  Solve  leadership  problems  —  identify  and  solve 
leadership/supervision  problems  (e.g.,  involving  dis¬ 
cipline,  training  needs,  conflicts  between  people, 
motivation  of  team  members). 

31.  Solve  medical  problems  —  identify  and  solve  physiral 
health  problems  (i.e.,  problems  that  require  special 
training  beyond  basic  first  aid). 

32.  Solve  communication  problems  -  identify  and  solve 
communication  problems  (i.e.,  not  technical/equipment 
problems,  but  problems  resulting  from  inaccurate  or 
non-extistent  commuinication). 

Operate  Equipment 

33.  Operate  precision  hand-held  equipment  ~  operate 
hand-held  equipment  requiring  great  precision  (e.g., 
syringe,  calipers,  soldering  or  welding  equipment). 

34.  Operate  hand-held  tools  ~  operate  hand-held  equip¬ 
ment  that  does  not  require  great  precision  (e.g.,  ham¬ 
mer,  wrench). 

35.  Operate  hand-held  power  equipment  -  operate  hand¬ 
held  power  assisted  equipment  (e.g.,  electric  saw, 
electric  wrench). 

36.  Operate  large  power  equipment  --  operate  large  power 
assisted  equipment  (e.g.,  forklift,  bulldozer,  backhoe). 
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37.  Operate  full  keyboard  --  operate  full  typewriter  or 
computer  keyboard. 

38.  Operate  numeric  keyboard  --  operate  a  numeric 
keyboard  (i.e.,  just  10  basic  keys,  0-9). 

Adjust  and  Control 

39.  Adjust  control  device  using  one  limb  —  make 
adjustments  in  control  devices  by  using  just  one  hand 
or  one  foot  (e.g.,  a  dial,  a  lever,  or  a  foot  pedal). 

40.  Adjust  control  device  using  multiple  limbs  ~  make 
adjustments  in  control  devices  by  using  more  than  one 
limb  (e.g.,  twisting  a  cylinder  with  one  hand  while 
setting  a  dial  with  the  other,  or  depressing  a  footpedal 
while  moving  equipment  into  proper  alignment  with 
hands). 


Drive 

41.  Drive  tracked  vehicle  --  drive  heavy  vehicles  (tracked). 

42.  Drive  heavy  wheeled  vehicle  --  drive  heavy  vehicles 
(wheeled). 

43.  Drive  light  wheeled  vehicle  —  drive  light  wheeled 
vehicles. 
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Aiming 


44  Aim:  stationary  target  -  aim  at  a  stationary  object. 

45.  Aim:  moving  target  --  manually  track  a  moving  target. 

Ph;^ical  Actions 

46.  Walk  long  distances  --  walk  long  distances  carrying 
significant  weight. 

47.  Run  short  distances  --  run  short  to  middle  distances. 

48.  Push,  pull,  lift  heavyweights  -  push,  pull,  or  lift 
heavy  weights. 

49.  Throw  objects  -  throw  hand-held  objects. 

50.  Sort  fold,  feed  by  hand  -  use  hands  to  fold  objects, 
sort  objects,  or  feed  objects  into  a  machine. 

5 !•  Make  coordinated  movements  -  use  well  coordinated 
hand,  arm,  and  upper  body  movements  (e.g.,  as  in 
setting  up  equipment,  or  moving  hazardous  material 
quickly). 

52.  Work  long  hours  --  work  long  hours  without  rest. 

53.  Work  under  adverse  conditions  --  work  under  adverse 
or  danagerous  conditions. 
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Name 


JOB  ACTIVITY  RATING  EXERCISE 
EVALUATION  SHEET 


Job  Activity  Definitions 


1.  How  clear  were  the  job  activity  definitions? 


12  3 


4  5 


Not  at  all 
clear 


Neither  clear 
nor  unclear 


6  7 

Very  clear 


2.  How  well  did  you  understand  the  terms  used  in  the  job  activity 
definitions? 


There  were 
many  terms 
I  did  not 
understand 


There  were 
some  terms 
I  did  not 
understand 


There  were 
no  terms 
I  did  not 
understand 


3.  Which  job  activity  was  most  difficult  to  judge? 


4.  Which  job  activity  was  least  difficult  to  judge?  _ 

5.  What  percentage  of  your  job  was  covered  by  these  activities? 

0%  10%  20%  30%  40%  50%  60%  70%  80%  90%  100% 


If  you  answered  less  than  100%,  what  activities  should  be  added? 


CIO 


Job  Activity  Ratings 


1.  How  clear  were  the  instructions  for  making  the  ratings? 

1  2  3  4  5  6  7 

Not  at  all  Neither  clear  Very  clear 

clear  nor  unclear 


2.  Were  the  ratings  easy  or  difficult  to  make? 

1  2  3  4  5  6  7 


Very 

difficult 


Neither  easy 
nor  difficult 


Very 

easy 


3  *  How  confident  did  you  feel  about  the  accuracy  of  your  ratings? 


Not  at  all 
confident 


Somewhat 

confident 


Very 

confident 


General  Evaluation 

1.  Which  of  the  following  would  you  change  (check  all  that  apply): 

_  Job  Activity  Definitions 

■  Job  Activity  Rating  Instructions  and  Procedures 


2 .  What  changes  would  you.  make? 


HATCHING  EXERCISE  INSTRUCTIONS: 
MOS  TASKS  AND  JOB  ACTIVITIES 


, this  portion  of  the  workshop,  we  ere  interested  in 
finding  out  how  well  our  list  of  53  job  activities  can  be  matched 
up  with  tasks  from  the  Soldier's  Manual  (SM)  and  from  the  Army 
Occupational  Survey  Program  (AOSP)  task  list. 

We  have  reviewed  the  SM  and  the  AOSP  and  have  selected  30 
tasks  that  cover  the  range  of  responsibilities  for  Skill  Level  1 
and  Skill  Level  2  soldiers.  Following  each  task,  there  are  three 
blank  lines.  In  those  blanks,  we  want  you  to  list  the  numbers  of 
up  to  3  job  activities  that  best  describe  that  task.  You  can 
find  the  nipbers  of  the  job  activities  on  the  "MOS  Job  Activity 
Questionnaire"  that  you  completed  during  the  previous  exercise. 
This  form  provides  the  number,  the  name,  and  a  brief  description 
of  each  of  the  53  job  activities.  Or,  if  you  would  prefer,  the 
30b  activity  names  and  numbers  also  can  be  obtained  from  the 
attached  handout,  "List  of  Job  Activities"  (which  does  not 
include  any  descriptions  of  the  activities) . 

As  you  review  the  job  activities,  first  find  the  job 
activity  that  best  describes  the  task.  Put  the  number  of  this 
30b  activity  in  the  first  colvimn,  under  "1st  Category/Activity. " 
Then  find  the  number  of  the  job  activity  that  provides  the  second 
best  description  of  the  task  and  mark  this  number  under  "2nd 
Category/Activity . "  Finally,  do  the  same  for  the  third  job 
activity. 

You  do  not  have  to  fill  in  all  three  blanks  for  all  of  the 
tasks.  You  may  find  that  only  one  or  two  job  activities  describe 
some  tasks.  Feel  free  to  leave  the  remaining  column (s)  blank. 

If  you  find  that  none  of  the  job  activities  describes  a  task 
simply  write  "NONE"  across  the  three  blanks. 

Also,  you  are  not  limited  in  the  number  of  times  that  you 
can  use  one  of  the  job  activities.  For  example,  if  the  same  job 
activity  describes  all  of  the  tasks  on  the  Task  List,  you  can  use 
that  activity  for  every  task. 
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List  of  Job  Activities 


1.  Work  in  a  team 

2 .  Lead  a  team 

3 .  Support/ counsel  peers 

4 .  Support/ counsel  subordinates 

5.  Coach  peers 

6 .  Coach  subordinates 

7.  Make  oral  reports  (to  individuals) 

8.  Make  oral  reports  (to  groups) 

9.  Relay  oral  instructions 

10.  Ask  questions 

11.  Record  information 

12 .  Write  brief  messages 

13.  Write  longer  reports 

14.  Monitor/ interpret  verbal  messages 

15.  Recall  verbal  information 

16.  Monitor/ inte^ret  nximerical  information 

17 .  Recall  numerical  information 

18.  Monitor/ interpret  figural  information 

19.  Recall  figural  information 

20.  Follow  oral  directions 

21.  Follow  written  directions 

22.  Judge  size  and  distance 

23.  Judge  location 

24.  Judge  paths  of  moving  objects 

25.  Solve  electrical  system  problems 

26.  Solve  mechanical  system  problems 

27.  Solve  logistical  problems 

28.  Solve  tactical  maneuver  problems 

29.  Solve  administrative  problems 

30.  Solve  leadership  problems 

31.  Solve  medical  problems 

32.  Solve  communication  problems 

33.  Operate  precision  hand-held  equipment 

34.  Operate  hand-held  tools 

35.  Operate  hand-held  power  equipment 

36.  Operate  larger  power  equipment 

37.  Operate  full  keyboard 

38.  Operate  numeric  keyboard 

39.  Adjust  control  device  using  one  limb 

40.  Adjust  control  device  using  multiple  limbs 

41.  Drive  tracked  vehicle 

42.  Drive  heavy  wheeled  vehicle 

43.  Drive  light  wheeled  vehicle 

44.  Aim:  stationary  target 

45.  Aim:  moving  target 

46.  Walk  long  distances 

47.  Run  short  distances 

48.  Push,  pull,  lift  heavy  weights 

49.  Throw  objects 

50.  Sort,  fold,  feed  by  hand 

51.  Make  coordinated  movements 

52 .  Work  long  hours 

53 .  Work  under  adverse  conditions 
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NAME 


TASK  MATCHING  EXERCISE 

Task  List  for  llB 
Form  A 


Task 

No.  Task  Name 


1st  2nd 

Task  Task 

Category/  Category/ 
Activity  Activity 


1.  Prepare  a  dragon  for  firing 

2.  Techniques  of  movement  in 
urban  terrain 

3.  Call  for/adjust  indirect 
fire 

4.  Select  fire  team/ scout  squad 
overwatch  position 

5.  Set  headspace  and  timing"  on 
a  caliber  .50  machinegun 

6.  Place  an  AN/PVS-5  (night 
vision  goggles)  into  opera¬ 
tion 

7.  Zero  AN/PVS-4  to  an  M16A1 
rifle 

8.  Select  hasty  firing  posi¬ 
tions  in  urban  terrain 

9.  Operate  radio  set  AN/PRC-77 
or  AN/PRC-25  (AN/GRC-160  or 
AN/GRC-125  assembled  for 
manpack  operations) 

10.  Collect/ report  information  - 
-SALUTE 

11.  Recognize  and  identify 
friendly  and  threat  armored 
vehicles 


3rd 

Task 

Category/ 

Activity 
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Task 

No.  Task  Name 


1st  2nd 

Task  Task 

Category/  Category/ 
Activity  Activity 


13.  Identify  terrain  features  on 
a  map 

14.  Perform  operator  maintenance 
on  an  M16A1  rifle,  magazine, 
and  ammunition 

15.  Load,  reduce  a  stoppage,  and 
clear  an  M60  machinegun 

15 •  Prepare  a  range  card  for  an 
M60  machinegun 

17 .  Engage  targets  with  an  M72A2 
LAW 

18.  Engage  enemy  targets  with 
hand  grenades 

19.  Install  and  fire/recover  an 
M18A1  Claymore  mine 

20.  Move  under  direct  fire 

21.  Move  over,  through,  or 
around  obstacles  (except 
minefields) 

22.  Camouflage  yourself  and  your 
individual  equipment 

23.  Put  on,  wear,  and  remove 
your  M17 -series  protective 
mask  with  hood 

24.  Put  on  and  wear  protective 
clothing 
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3rd 

Task 

Category/ 

Activity 


1st  2nd 

Task  Task 

Category/  Category/ 
Activity  Activity 


25.  Perform  cardiopulmonary 
resuscitation  (CPR)  on  an 
adult  using  the  one-man 
method 

26.  Put  on  a  field  or  pressure 
dressing 

27 .  Navigate  from  one  point  on 
the  ground  to  another  point 

28.  Establish  an  observation 
post 

29.  Conduct  day  and  night 
surveillance  without  the 
aid  of  electronic  devices 

30.  Administer  nerve  agent  an¬ 
tidote  to  self  (self-aid) 


Task 

No.  Task  Name 


cie 


3rd 

Task 

Category/ 

Activity 


NAME 


TASK  MATCHING  EXERCISE 

Task  List  for  63B 
Form  A 


Task 

No.  Task  Name 


1st  2nd  3rd 

Task  Task  Task 


Category/  Category/  Category/ 
Activity  Activity  Activity 


1.  Adjust  clutch  pedal  free 
travel 

2.  Maintain  assigned  toolkit 

3 .  Repair  electrical  wiring 
(truck,  cargo,  1-1/4  ton, 
4x4} 

4 .  Replace  air  hydraulic 
cylinder 

5.  Replace  radiator  (truck, 
cargo,  2-1/2  ton,  6x6) 

6.  Replace  service  brakes 
(truck,  utility,  1/4  ton, 
4x4) 

7.  Replace  fuel  pump  (truck, 
cargo,  2-1/2,  ton,  6x6) 

8 .  Replace  wheel  bearings 
(truck,  cargo,  2-1/2  ton, 
6x6) 

9.  Tow  disabled  vehicle  with  5 
ton  wrecker 

10.  Perform  annual  PMCS  (inspect 
and  road  test)  (truck,  car¬ 
go,  5  ton,  6x6) 

11.  Troubleshoot  brake  system 
malfunctions  (truck,  cargo, 
1-1/4  ton,  4x4) 
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1st  2nd 

Task  Task 

Category/  Category/ 
Activity  Activity 

12 .  Troxibleshoot  electrical  _  _ 

system  (truck,  cargo,  5  ton, 

6x6) 

13 .  Troxibleshoot  fuel  system  _ 

malfunctions  (truck,  cargo, 

2-1/2  ton,  6x6) 

14.  Troubleshoot  service  brake  _ 

malfunctions  (truck, 

utility,  1/4  ’ton,  4x4) 

15.  Replace  starter  _  _ 


16.  Administer  first  aid  to  a 
nerve  agent  casualty  (buddy- 
aid) 

17 .  Camouflage  equipment 


18.  Perform  operator  maintenance 
on  an  M16A1  rifle,  magazine, 
and  ammunition 

19.  Determine  grid  coordinates 


20.  Detezrmine  magnetic  azimuth 
using  compass 

21.  Load,  reduce  stoppage, 
clear  M16A1  rifle 

22.  Use  challenge  and  password 


23.  Put  on  and  wear  M17  series 
protective  mask  with  hood 


Task 

No.  Task  Name 


3rd 

Task 

Category/ 

Activity 
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Analysis  of  the  Relationship  Between  SQT  and 
Aptitude  Area  Composite  Scores  for 
Transportation  Corps  MOS 


Jane  M.  Arabian 

U.S.  Army  Research  Institute 
for  the  Behavioral  and  Social  Sciences 


The  purpose  of  this  research  was  to  examine  the  relationship 
between  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB) 
aptitude  area  (AA)  composite  scores  and  Skill  Qualification  Test 
(SQT)  scores.  The  research  was  undertaken  to  provide  the  U.S.  Army 
Transportation  School  with  information  about  the  job  proficiency, 
given  current  AA  entry  standards,  of  soldiers  in  selected  Transpor¬ 
tation  Corps  military  occupation  specialities  (MOS). 

The  nine  AA  composites  currently  used  by  the  .Army  to  select  and 
classify  enlisted  personnel  are  created  by  combining  ASVAB  subtest 
scores.  Aptitude  a^ea  cut-off  scores  are  used  at  the  time  of  enlist¬ 
ment  to  determine  eligibility  for  entering  given  MOS.  Minimum  AA  scores 
are  established  for  MCS  to  increase  the  likelihood  that  individuals 
allowed  entry  to  an  MOS  will  be  able  to  perform  their  duties. 

In  order  to  ascertain  how  effectively  an  AA  cut-off  functions  as 
an  entrance  standard,  some  measure  of  job  performance  is  needed.  The 
written  SQT,  which  is  now  part  of  the  Army  Individual  Training  Evalu¬ 
ation  Program  (ITEP),  is  the  best  available  measure  of  MOS-specific 
job  knowledge.  The  .Army  has  administered  SQT  to  enlisted  soldiers 


since  1977,  and  has  established  the  score  of  60  points  or  better  as 
acceptable  (passing). 

If  the  majority  of  soldiers  with  the  minimum  AA  score  in  a  par¬ 
ticular  MC6  pass  the  SQT  then  the  job  proficiency  of  soldiers  within 
that  MOS  could  be  seen  as  satisfactory.  Cn  the  other  hand,  if  the 
majority  of  soldiers  meet  the  MOS  AA  cut-off  but  fail  the  SQT  then 
the  entrance  standards  may  need  to  be  changed.  In  such  cases,  it 
may  be  worthwhile  to  consider  making  entry  into  the  MOS  more  difficult 
Raising  minimum  AA  scores  would  be  one  option  for  improving  soldiers'' 
proficiency  levels  in  an  MOS. 

METHOD 

Sample .  U.S.  Army  Transportation  School  MOS  were  analyzed  in 
this  research.  To  permit  meaningful  analysis,  each  MOS  was  required 
to  have  at  least  100  soldiers  with  both  ASVAB.  and  SQT  data  available. 
The  two  MOS  meeting  this  requirement  were:  57H  (Cargo  Specialist, 
n=183)  and  64C  (Motor  Transport  Operator,  n=2038).  A  third  MOS,  71N 
(Traffic  Management  Coordinator),  was  also  analyzed  even  though  the 
sample  size  was  under  100  (n=92) .  Interpretation  of  the  data  for  71N 
should,  therefore,  be  made  with  caution.  All  the  soldiers  in  the 
three  MOS  entered  the  Army  between  October  1980  and  September  1982. 

Predictors.  Each  MOS  in  this  research  sample  employed  a  differ¬ 
ent  ASVAB  AA  composite  as  an  entrance  standard.  Therefore,  a  differ¬ 
ent  AA  composite  score  was  used  as  the  predictor  measure  for  each  MOS 
A  minimum  score  of  85  on  the  General  Maintenance  (GM)  AA  composite 


is  required  for  entry 'into  57H.  The  GM  composite  is  made  up  of 
scores  from  general  science  (GS),  auto/shop  information  (AS),  mathe¬ 
matics  knowledge  (MK),  and  electronics  information  (El)  ASVAB  subtests. 
Soldiers  entering  64C  are  required  to  have  a  score  of  at  least  85  on 
the  Operators/Food  (OF)  AA  composite.  Ihe  OF  composite  is  formed  by 
combining  scores  from  verbal  ability  (VE),  number  op^-ations  (NO), 

I 

auto/shop  information  (AS),  and  mechanical  comprehension  (MC)  subtests. 
A  minimum  score  of  95,  on  the  Qerical  (CL)  composite  (currently  made 
up  of  verbal  ability  (VE) ,  number  operations  (NO),  and  coding  speed 
(CS)  ASVAB  subtests)  is  required  for  71N. 

Criteria .  Scores  obtained  on  the  SQT  by  soldiers  in  the  selec¬ 
ted  MOS  served  as  the  criterion  measure.  The  SQT  scores  were  obtained 
from  tests  administered  during  the  first  two  quarters  of  the  1983 
testing  year. 

Analysis.  Two-way  probability  tables  were  calculated  for  AA 
composite  and  SQT  scores.  The  composite  score  ranges  were  set  at 
5-point  intervals,  starting  at  the  current  cut-off  score  for  each 
MOS,  with  a  separate  interval  for  all  scores  at  or  above  110.  Five- 
point  intervals  were  chosen  as  the  basic  AA  category  range  because 
the  Army's  existing  classification  system  establishes  cut-off  scores 
in  5-point  intervals.  Four  SQT  categories  were  created:  below  60 
(failing),  greater  than  or  equal  to  60,  70,  and  80. 


RESULTS  AND  DISCUSSION 


The  data  for  57H,  64C,  and  71 N  are  presented  in  Tables  1-3, 


TABLE  1 


Probability  of  Obtaining  Given  SQT  Scores 
by  Aptitude  Area  Composite  Score 
(57H) 


GM 

SQT  Score  Aptitude  Area  Composite  Score  Category 


Category 

>  85 

>  90 

>  95 

>  100  '  > 

105 

>  110 

>  80 

15 

14 

16 

14 

27 

38 

>  70 

45 

44 

47 

43 

45 

50 

>  60 

80 

82 

87 

81 

82 

88 

<  60 

20 

18 

13 

19 

18 

12 

Cumulative 

183 

83 

38 

21 

11 

8 

n 


4 


TABLE  2 


I 

Prdbability  of  Obtaining  Given  SQT  Scores 
by  Aptitude  Area  Composite  Score 
(640 


OF 


SQT  Score 
Category 

Aptitude  Area  Composite  : 
>  85  >90  >95  >  100 

Score  Category 
>  105  >  110 

>  80 

59  , 

64 

69 

72 

77 

.  80 

>  70 

90 

92 

93 

95 

95 

96 

>  60 

98 

98 

98 

99 

99 

99 

<  60 

2 

2 

2 

1 

1 

1 

Cumulative 

2038 

1444 

1037 

700 

453 

319 

n 


TABLE  3 


Probability  of  Obtaining  Given  SQT  Scores 
by  Aptitude  Area  Composite  Score 
. (71N) 


CL  ^ 

SQT  Score  Aptitude  Area  Composite  'Score  Category 


Category 

>  95 

>^100 

>  105 

>  110 

>  80 

61 

60 

74 

72 

>  70 

80 

79 

89 

88 

>  60 

92 

92 

91 

92 

<  60 

8 

8 

9 

8 

Cumulative 

n 


92 


62 


35 


25 


respectively.  For  example,  Table  1  indicates  that  for  the  183 
57H  soldiers  who  have  GM  AA  scores  greater  than  or  equal  to  85,  80? 
obtained  an  SQT  score  of  60  or  better.  The  probability  of  obtaining 
an  SQT  score  of  60  or  better  was  only  2%  greater  for  soldiers  having 

AA  scores  of  90  or  more  (82  minus  80). 

It  can  be  seen  that,  in  general,  as  AA  scores  increase,  the 
probability  of  obtaining  a  particular  passing  SQT  score  also  in¬ 
creases  for  all  the  MOS  in  this  research  sample.  This  positive 
relationship  between  AA  and  SQT  scores  does  not  appear  to  be  dramatic 
because  of  a  "ceiling  effect"  (i.e.,  the  fact  that  a  vast  majority  of 
soldiers  in  each  MOS  with  at  least  the  lowest  acceptable  AA  composite 
score  pass  the  SQT).  Anoraolies  in  the  data,  such  as  a  decrease  in  the 
probability  of  obtaining  a  given  SQT  score  from  one  AA  category  to  the 
next  (higher)  AA  category,  are  most  likely  a  result  of  the  relatively 
small  sample  sizes. 

Overall,  given  the  high  probability  of  obtaining  an  SQT  score  of 
60  or  better  with  an  AA  score  at  or  above  the  entry  standard  .(80?, 

98?,  and  92?  for  57H,  64C,  and  71N,  respectively),  the  data  would  not 
support  a  recomnendation  to  increase  the  AA  cut-off  scores .  Further¬ 
more,  in  the  present  sample  57H,  which  has  the  highest  failure 
probability  at  the  lowest  end  of  the  AA  scores  (20?),  an  increase  of 
five  points  in  the  AA  cut-off  score  would  result  in  only  a  modest  2? 
increase  in  the  probability  of  passing  the  SQT .  This  probable  2? 
performance  increase  would  be  achieved  at  the  cost  of  eliminating  from 
eligibility  a  large  proportion  of  enlisted  soldiers. 


Based  on  the  data  obtained  in  this  research,  the  cost  of  eliminating 
soldiers  does  not  seem  to  be  worth  the  benefit  of  a  2%  improvement 
in  jDerformance  as  measured  by  the  SQT. 
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Analysis  of  TRASANA  Vfritten  Performance  Test  Scores  as  a 
Function  of  Aptitude  Area  Composite  Scores  for 
Logistics  MOS 

Jane  M.  Arabian 
Army  Research  Institute 
for  the  Behavioral  and  Social  Sciences 

The  U.S.  Army  Quartermaster  School  requested  the  Army  Research 
Institute  to  examine  the  relationship  between  the  Armed  Services  Voca¬ 
tional  Aptitude  Battery  (ASVAB)  aptitude  area  (AA)  composite  scores  and 
soldier  performance.  Previous  research  (Rossmeissl  &  Eaton,  1984)  re¬ 
ported  on  relationships  between  AA  scores  and  Skill  Qjalification  Test 
(SQT)  performance  in  some  Quartermaster  Military  Occupational  Specialties 
(MOS):  76c,  76V,  and  94B.  With  the  cooperation  of  TRASANA,  the  research 
reported  in  this  paper  employed  scores  on  written  job  skill  and  knowledge 
tests  developed  and  administered  by  TRASANA  as  a  measure  of  soldiers’ 
job  performance.  Ihe  TRASANA  test  items  were  derived  from  soldier 
manuals  and  presented  in  a  multiple  choice  format  for  each  MOS..  These 
tests  were  developed  independently  of  the  SQT  which  also  measures  a 
soldier’s  on-the-job  performance  and  has  been  administered  by  the 
Army  since  1977. 

The  purpose  of  examining  the  relationship  between  AA  composite  scores 
(predictor  measure)  and  measures  of  job  performance  (criterion  measure) 
is  to  obtain  information  for  setting  AA  minimum  (cut-off)  scores.  Aptitude 
area  cut-off  scores  are  used  to  determine  eligibility  for  entry  into  MCS. 
The  nine  AA  composites  currently  used  by  the  Army  to  select  and  classify 
enlisted  personnel  are  created  by  combining  the  ASVAB  subtests’  scores. 


Aptitude  area  mlnltnun  scores  are  established  for  each  HOS  to  Increase  the 
likelihood  that  Individuals  who  are  allowed  entry  to  an  MOS  will  be  able 
to  perform  their  jobs. 

METHOD 

Sample.  Four  U.S.  Army  Quartermaster  School  MOS  were  analyzed  in 
this  research.  To  permit  meaningful  analysis,  each  MOS  was  required  to 
have  at  least  100  soldiers  with  both  ASVAB  .and  TRASANA  test  score  data 
available.  The  four  MOS  meeting  this  requirement  were:  76C  (Equipment 
Records  and  Parts  Specialist,  n=476),  76P  (Materiel  Control  Accounting 
Specialist,  n=386),  76V  (Materiel  Storage  and  Handling  Specialist,  t>=  570), 
and  76Y  (Unit  Supply  Racialist,  n=972).  Ihe  data  for  this  research  came 
from  soldiers  in  these  MOS  who  were  tested  by  TRASANA  in  1983/1984. 

Predictors.  All  the  MOS  in  the  research  sample  use  the  clerical 
(CL)  composite  as  the  standard  for  enlistment  eligibility.  Therefore, 

CL  scores  were  used  as  the  predictor  variable  in  this  research.  The  CL 
composite  is  currently  formed  by  combining  the  verbal  ability  (VE).,  numer¬ 
ical  operations  (NO),  and  coding  speed  (CS)  ASVAB  subtests. 

nritcria.  The  criterion  measure  for  this  research  was  the  per  cent 
correct  score  on  the  TRASANA  written  job  skill  and  knowledge  test.  The 
criteria  data  were  tabulated  for  soldiers  who  also  had  ASVAB  data  available. 

Analyses.  Since  TRASANA  was  not  allowed  to  release  their  raw  data  they 
prepared  two-way  frequency  distribution  tables  of  AA  composite  and  written 
test  scores  for  the  Army  Research  Institute  (ARI)  to  analyze.  The  TRASANA 


tables  were  irodified  (i.e.,  cells  were  pooled)  to  conform  more  closely  with 
previous  ARI  research.  Frequency  and  cumulative  percentage  data  were  then 
derived  from  the  revised  tables.  Composite  score  range  was  set  at  5-point 
intervals,  starting  at  the  current  cut-off  score  for  each  MOS,  and  with  a 
separate  category  for  all  scores  below  the  cut-off  and  at  or  above  110 
points.  Five-points  was  chosen  as  the  basic  AA  category  range  because 
the  Army’s  existing  classification  system  establishes  cut-off  scores  in 
5-point  intervals.  Since  there  is  no  passing  score  established  by 
the  Army  for  the  TRASANA  tests ,  category  ranges  for  these  scores  were 
set  arbitrarily  for  convenience.  With  the  exception  of  the  lowest  cat¬ 
egory  (below  19%  correct),  category  ranges  were  set  at  15-point  intervals. 

RESULTS  AND  DISCUSSION 

The  data  for  each  MOS  are  presented  in  probability  tables 
(Tables  1-4  for  76C,  76P,  76V,  and  76Y,  respectively). 

Overall,  there  appears  to  be  a  positive  relationship  between  AA 
scores  and  TRASANA  written  test  performance  in  the  four  MOS.  In  general, 
as -AA  score  catetories  increase,  test  performance  also  tends  to  improve. 
Departures  from  this  trend  may  be  due  to  the  relatively  small  sample  sizes. 
With  respect  to  the  setting  of  AA  cut-off  standards,  the  data  from  the 
four  MOS  are  someviiat  difficult  to  interpret.  The  reason  for  this 
difficulty  is  discussed  below. 

Unlike  SOT  which  has  an  established  passing  score  of  60,  the  TRASANA 
test  does  not  have  a  set  passing  grade.  This  is  what  primarily  makes  data 
interpretation  difficult.  Is  50%  correct  on  the  TRASANA  test  satisfactory 


TABLE  1 


Probability  of  Obtaining  a  Written  Test  Score 
Given  Aptitude  Area  Composite  Score 
(760 


%  Correct 
On  TRASANA 

Written  Test  >.  95 


CoiDposite  Score 

100  >105  >.  110 


>  80 

.18 

>  65 

.67 

>  50 

oo 

GO 

• 

>  35 

.96 

>  20 

.98 

<  20 

.02 

(n) 

(302) 

.2H 

.26 

.27 

.69 

-.77 

.79 

.88 

.91 

.95 

.96 

.95 

.97 

.98 

.98 

1.00 

.02 

.02 

.00 

(208)  (129)  -  (77) 


TABLE  2 


Probability  of  Obtaining  A  Written  Test  Score 
Given  Aptitude  Area  Composite  Score 
(76P) 


f  Correct 

On  TRASANA 
Written  Test 

>  90 

CL  Composite  Score 
>^95  >  100 

>  105 

>  110 

>  80 

.05 

.08 

.11 

.15 

.21 

>65 

.28 

.37 

.41 

.51 

.62 

>  50 

.65 

.73 

.76 

.80 

.81 

>  35 

.92 

.93 

.94 

.94 

.92 

>  20 

.99 

.99 

1.00 

1.00 

1.00 

<  20 

.01 

.01 

.00 

.00 

.00 

(n) 


(301)  (157) 


(111) 


(7^) 


(53) 


TABLE  3 


Probability  of  Obtaining  a  Written  Test  Score 
Given  Aptitude  Area  Composite  Score 
(76V) 


%  Correct 

On  TRASANA 
Written  Test 

>  90 

CL  Composite  Score 
>^95  >.  100 

>  105 

>  110 

>  80 

.00 

.00 

.00 

.00 

.03 

>  65 

.06 

.09 

.  .12 

.14 

.18 

>  50 

.37 

.n6 

.53 

.54 

.60 

>  35 

.76 

.78 

.79 

.81 

.82 

>  20 

.97 

.98 

.98 

.97 

.99 

<  20 

.03 

.02 

.02 

.03 

.01 

(n) 

(tl22) 

i2‘\0) 

(168) 

-  (110) 

(77) 

TABLE  H 


Probability  of  Obtaining  a  Written  Test  Score 
Given  Aptitude  Area  Composite  Score 
(76Y) 


f  Correct  „ 

On  TRASANA  CL  Composite  Score 

Written  Test  >95  >  100  >  105 


>  80 

.00 

>  65 

.18 

>  50 

.62 

>  35 

.93 

>  20 

1.00 

<  20 

.00 

(n) 

(861) 

.00 

.01 

.00 

.20 

.23 

.26 

.66 

.69 

.74 

.9^ 

.95 

.94 

1.00 

1.00 

1.00 

.00 

.00 

.00 

(593) 

(393) 

(254) 

or  ’'passing"?  What  about  65%  or  35%  correct?  If  50%  correct  is  conserv¬ 
atively  taken  for  a  passing  score  then,  of  the  total  number  of  soldiers 
in  each  MGS  with  AA  scores  at  or  above  the  MOS-specific  cut-off  point, 

88%  of  76C,  65%  of  76P,  37%  of  76V,  and  62%  of  76Y  pass  their  TRASANA 

written  skill  and  knowledge  test. 

Looking  at  each  MOS  more  closely.  Table  1,  for  example, 
indicates  that  there  is  an  88%  chance  that  a  760  soldier  with  an 
AA  score  at  or  above  95  will  score  50%  or  better  on  the  TRASANA  written 
test.  However,  a  soldier  with  an  AA  score  at  or  above  100  is  as  likely 
as  a  soldier  with  an  AA  score  at  or  above  95  to  obtain  at  least  50% 
correct  on  the  TRASANA  test.  In  other  words,  for  760,  increasing  the  AA 
cut-off  by  five  points  does  not  improve  the  likelihood  of  soldiers  scoring 
more  than  50%  on  the  skill  and  knowledge  test.  The  data  are  similar  for 
76Y.  Increasing  the  AA  cut-off  fVom  95  ot  100  only  increases  the  chance  of 
scoring  50%  or  better  on  the  TRASANA  test  by  (from  .62  to  .66). 

The  probability  data  for  76P  and  76V  (Tables  2  and  3,  respectively) 
indicate  a  different  situation.  Increasing  the  cut-off  standard  from 
90  to  95  for  these  MOS  improves  the  probability  that  soldiers  would  score 
at  least  at  the  50%  level  on  the  skill  and  knowledge  test,  There  is  an 
8%  increase  in  probability  for  76P  and  a  9%  increase  for  76V.  (However, 
if  a  65%  passing  score  is  accepted  for  the  TRASANA  test  then  raising  the 
AA  cut-off  standard  results  in  a  9%  improvement  in  performance  only  for 

76P.') 

While  these  data  indicate  that  76P  and  76V  could  possibly  benefit 
from  a  5-point  increase  in  their  respective  AA  entry  standards, -the 


possible  benefits  must  be  weighed  against  the  likely  exclusion  of  a 
large  number  of  enlisted  soldiers  from  these  two  MOS.  aher  information 
about  the  MOS  in  this  research  should  also  be  taken  into  account  before  pol¬ 
icy  decisions  are  made.  For  instance,  Rossraeissl  and  Eaton  included  760 
and  76V  in  their  research.  That  research,  using  SQT  score  as  the  criter¬ 
ion  measure,  suggested  that  increasing  the  AA  cut-off  for  760  but  not  76V 
merited  farther  consideration.  The  present  research  with  the  TRASANA 
t6sts  sugg6sts  quite  the  opposite* 

The  differences  between  the  results  of  the  two  research  reports  may  be 
due  to  any  one  or  more  of  the  following  factors:  different  soldiers  in  the 
MOS  samples,  differences  between  the  content  of  the  SQT  and  TRASANA  tests, 
sample  sizes,  etc.  A  likely  source  of  the  difference  in  results  is  the 
difference  in  the  content  and  scaling  of  the  SQT  and  TRASANA  tests. 

Problems  of  this  nature  will  likely  continue  until  standard  performance 
measures  in  each  MCS  are  developed  and  accepted  for  personnel  decision¬ 


making  . 
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of  Job  perforMnc  1.  the  criterion  verleble. 
Heverthele...  In  ««  eltnetlon.  «.  Inaicetlo.  of  the 
dletrlbutlon  of  eoldter  qoellty  In  ter«  of  iFOI  percentile 
cetegorle.  1.  needed.  Since  dFQT  d.t.  ere  not  nnleereell, 
.yelleble  to  Anw  the  conver.lon  teble.  nere  developed. 

The  etteched  teblet,  one  for  eech  eptltnde  eree  (AA) 
co.po.lte,  ere  epproprl.t.  for  nee  -rtth  AA  .cor«.  fro.  ASVAB 
11/12/13/1*  end  other  ASVAB  AA  .cores  .hlch  heve  been  converted 
to  the  H80  youth  populetlon  nor.  group.  The  tebl.s  cen  be  ueed 
to  convert  AA  etenderd  .core  distribution.  Into  expected  AFQI 

cetegory  distribution..  The  conver.lon  teble.  uer.  clculeted 

fro.  cro..  Mbulatlon.  (frequency  distribution.)  of  AA  .tenderd 
scores  by  ATOT  percentile  scores.  The  Iroflle  of  Aserlce.  touch 
(none, 1980)  dote  Mt  uas  used  to  derive  the  tebles.  The  flnsl 

none  .«^.l..  based  on  9,173  cses,  contains  ASVAB  data  fro.  a 

representative  Muple  of  18-23  year  olds  In  the  population;  the 
corresponding  popuUtlon  .Is.  (.sighted  «..pl.)  !•  ««  ” 


million* 

In  order  to  use  the  tables,  first  obtain  sample  sizes  for 
the  5-point  aptitude  area  score  ranges  on  the  AA  composite  used 
for  the  MOS's  entrance  standard.  Second,  multiply  the  sampl 
size  of  each  AA  score  range  by  the  percentages  in  the  appropriate 
column  for  the  AA  score  range.  This  results  in  the  expected  AFQT 
distribution  for  the  Individuals  in  the  sample.  For  example,  if 
100  Individuals  scored  between  90  and  94  on  the  CL  AA  composite 
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then  you  would  expect  6.11  Individual*  (100  x  .0611)  to  he  in  the 
AFQT  I-IIIA  category.  76.19  (100  x  .7619)  to  he  in  the  IIIB 
category.  17.70  (100  x  .1770)  to  he  in  the  IV  category,  and  no 
one  to  fall  in  the  V  category. 

A  sample  application  of  the  conversion  tahles  ia  alao 
appended  to  illustrate  further  bow  the  tahles  are  used.  The 
example  is  based  on  a  fictitious  ssi  of  data. 
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CONVERSION  TABLES  FOR  ESTIMATING  AFQT  PERCENTILE  DISTRIBUTIONS  FROM  APTITUDE 
AREA  STANDARD  SCORE  DISTRIBUTIONS 
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Aptitude  Area  Couposlte 


Arts  Co»po®^^* 


100*001  100*00 


HN  Aptltud*  *rM  Co«po8lt« 
AA  Standard  Seora 


Aptitude  Are*  Co*poalte 
Standard  Soor* 
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Coaposit# 


Aptitude  Area  Coupoaite 


SAMPLE  APPLICATION  OF  THE  AA  TO  AFQT  EXPECTED  DISTRIBUTION  TABLES 
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•  School  tequlte.  the  folloving  «:ceosloi>.  In  tecs  of  CL  ecotee 


AA  Score 
95-99 

n  A87 


Range 
100- lOA 

380 


105-109 

220 


110-114 

103 


•  Being  the  n*e  end  the  A»  to  driJT  dletrlhotlon  teble  for  the  CL 
composite,  the  follo%»ing  is  obtained: 


iAA  i  \ 

195-99  100-104  105-109  110-114}  } 

expected  }n  •  J 


AFQT 

distrib. 

1  487 

I 

1 

380 

220 

103 

}  Total 

1 

1 

S 

I-IIIA 

}135 

239 

194 

100 

}  66? 

56 

llIB 

!32B 

140 

26 

3 

}  497 

42 

IV 

i  24 

2 

0 

0 

!  26 

2 

NOTE:  The  numbers  in  the  table  have  been  rounded  off  to 

the  nearest  whole  number* 
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Introduction 

The  Army  Research  Institute  is  currently  engaged  in  a  large-scale, 
multi-year  research  project  to  improve  the  Army  selection  and  classifica¬ 
tion  system  (Project  A,  "Improving  the  Selection,  Classification  and  Uti¬ 
lization  of  Army  Enlisted  Personnel”)  and,  thereby,  increase  the  overall 
effectiveness  of  the  force.  The  research  is  aimed  at  developing  comprehen¬ 
sive  selection  and  classification  procedures  to  predict  validly  perform¬ 
ance  in  Army  training  and  occupational  specialties. 

A  number  of  performance  measures,  including  measures  of  training  suc¬ 
cess,  service— wide  performance,  and  MOS— specific  hands-on  performance, 
were  developed.  The  Army's  rationale  for  developing  multiple  measures  of 
job  performance  is  based  upon  the  knowledge  that  a  soldier's  job  is  multi¬ 
faceted  (i.e.,  many  different  kinds  of  tasks  are  involved)  and  there  are 
multiple  aspects  to  job  performance  (e.g.,  initiative,  obedience,  etc.). 
Therefore,  in  order  to  obtain  information  about  the  domain  of  job  perform¬ 
ance  behaviors,  the  Army's  research  project  has  developed  different  kinds 
of  tests  to  assess  these  different  aspects  of  job  performance.  Composite 


^The  opinions,  views  and  conclusions  contained  in  this  document  are  those 
of  the  author  and  should  not  be  interpreted  as  representing  the  official 
policies,  expressed  or  Implied,  of  the  U.S.  Army  Research  Institute  for 
the  Behavioral  and  Social  Sciences  or  the  Department  of  Defense  or  the 
United  States  Government. 


scores,  based  on  constructs  derived  from  the  performance  measures,  will  be 
used  as  indices  of  job  performance. 

Preliminary  analyses  of  field  test  data  and  other  research  lead  to  the 

'  > 

expectation  that  data  from  the  concurrent  validation  phase  of  the  project 
will  result  in  positive  correlations  between  predictor  and  criterion  meas¬ 
ures  of  performance.  This  information,  however,  will  not  lead  directly  to 
the  setting  of  enlistment  standards.  While  It  is  possible  to  use  cost 
trade-off  models  for  selection  and  classification  systems  once  a  perform¬ 
ance  objective  is  determined,  such  models  cannot  identify  the  required 
performance  objective.  Some  other  method  is  needed  to  define  performance 
requirements  or  standards  before  reasonable  enlistment  standards  can  be 
established . 

To  determine  whether  existing  methods  could  be  used  to  set  job  per¬ 
formance  standards  in  the  Army,  the  literature  review  in  Appendix  A  was 
conducted.  It  is  in  the  form  of  an  annotated  bibliography  summarizing  the 
content  of  each  reference.  While  the  bibliography  is  not  intended  to  be 
exhaustive,  it  is  representative  of  the  published  literature.  Table  1 
presents  a  listing  of  the  bibliographic  references  and  Indicates  the  broad 
categories  that  reflect  the  content  of  each  reference. 

Overview  of  Standard  Setting  Procedures;  General  Issues 

The  majority  of  the  references  present  definitions  of  standard  setting 
and  descriptions  of  various  procedures  that  have  been  developed.  The 
article  by  Glass  (1978)  describes  a  variety  of  standard  setting  methodolo¬ 
gies.  In  addition,  there  are  individual  references  for  methodologies 
developed  by  Angoff  (1971),  Jaeger  (1976,  1982,  198A)  and  Nedelsky  (1954). 
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Application 

(Specific) 


Table  1 
(Continued) 


Application 

(Specific) 


The  Poggio  (1984)  article  provides  a  good  comparison  of  various  procedures 
used  in  Kansas  to  set  standards  on  educational  competency  tests  for  read¬ 
ing  and  math.  A  large  group  of  references  deal  with  applications  of  stan¬ 
dard  setting  procedures.  By  far,  the  bulk  of  the  applications  deal  with 
setting  educational  standards  (e.g.,  minimally  acceptable  levels  of  read¬ 
ing  and  math  knowledge  for  high  school  graduates)  and  professional  certi¬ 
fication  (e.g.,  minimal  levels  of  knowledge  that  a  grammar  school  teacher 
must  possess  to  be  certified  or  licensed  to  teach  in  a  given  state).  There 
is  clearly  very  little  empirical,  applied  research  dealing  specifically 
with  the  determination  of  job  performance  standards  for  selection  and 
classification  purposes, 

standard  setting  for  certification  vs.  selection  and  classification. 

A  basic  difference  between  standards  for  competence  (mastery  vs.  non¬ 
mastery)  or  certification  (CC)  and  for  selection  and  classification  (SC) 
is  that  the  former  essentially  entail  only  one  judgement.  A  CC  standard  is 
used  to  indicate  that  an  individual  meets  the  qualifications  to  be  consid- 
ered  minimally  competent. 

Standard  setting  for  SC  purposes  requires  that  two  judgements  be  made. 
The  first  is  a  judgement  of  minimal  competence  or  acceptable  performance 
derived  through  measurement  of  job  performance  with  job  incumbents.  In 
other  words,  a  standard  needs  to  be  set  on  the  criterion  measure(s).  A 
second  cut-off  score  (standard)  needs  to  be  set  on  some  predictor  measure 
such  that  individuals  who  meet  the  standard  on  the  predictor  will  be  like¬ 
ly  to  meet,  at  some  later  point,  the  standard  for  accepUble  job  perform¬ 
ance.  Actually,  two  sets  of  these  types  of  judgements  may  be  needed:  One 
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for  selection  and  a  second  for  classification.  The  first  set  of  judge¬ 
ments,  for  selection,  may  require  different  considerations  than  the  set  of 
judgements  for  job  classification.  For  example,  selection  standards  may 
be  based 'on  concerns  regarding  supply  and  demand,  trainability  in  a  gen¬ 
eral  sense  and  attrition,  while  classification  standards  would  be  based 
upon  considerations  of  actual  job  performance.  The  Army  model  is  based  on 

this  sort  of  multiple  judgement  approach. 

Procedures;  judgement  and  validity.  Regardless  of  the  context  for 
setting  standards,  it  should  be  understood  that  the  application  of  any 
standard  setting  procedure  requires  judgement.  While  the  judgements  will 
be  value-laden,  they  are  not  therefore  arbitrary  in  the  sense  of  being 
based  solely  on  whimsey  (cf.  Hofstee,  1983).  The  judgemental  nature  of 
standard  setting  has  been  a  focus  of  debate  in  the  literature  (e.g.. 

Glass,  1978;  Hambleton,  1978).  However,  rather  than  dismiss  all  standard 
setting  procedures  because  they  require  judgements  we  need  a  more  con¬ 
structive  approach.  It  seems  reasonable  to  accept  the  fact  that  judge¬ 
ments  are  the  basis  of  standard  setting  procedures  and  then  examine  the 

validity  and  impact  of  the  resultant  cut-off  scores. 

Unfortunately,  the  literature  offers  very  little  guidance  for  select¬ 
ing  one  procedure  over  another  based  on  considerations  of  validity.  In¬ 
deed,  it  cannot  be  said  that  a  test  performance  standard  derived  from  any¬ 
one  procedure  is  intrinsically  valid  because  of  the  particular  procedure 
employed.  Andrew  and  Hecht  (1976)  found  that  different  groups  of  judges 
arrived  at  similar  sundards  using  any  one  procedure,  but  different  stan¬ 
dards  were  obtained  when  two  different  procedures  were  used  by  the  same 
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groups  of  judges.  According  to  Poggio's  (1984)  research,  different  proce¬ 
dures  will  consistently  yield  higher  or  lower  standards.  In  fact,  the 
validity  of  a  cut-off  score  (i.e.,  the  ability  of  the  cut-off  score  to 
discriminate  between  minimally  acceptable  and  unaccepuble  individuals)  is 
likely  to  depend  not  only  on  the  procedure  used  to  set  the  cut-off  but 
also  on  the  content  of  the  instrument(s)  used  to  assess  performance. 

Norm-referenced  vs  criterion-referenced  tests.  It  stands  to  reason 
that  if  a  test  does  not  suitably  measure  what  it  purports  to  measure  then 
any  standard  or  cut-off  score  based  on  that  test  will  not  be  valid.  This 
holds  for  tests  developed  within  either  a  norm-referenced  test  (NRT)  or 
criterion-referenced  test  (CRT)  framework.  Although  a  detailed  descrip¬ 
tion  of  NRT  and  CRT  development  procedures  will  not  be  presented  here, 
several  references  in  the  bibliography  (see  Table  1,  -Definitions")  dis- 
CUSS  the  methods  in  more  detail* 

One  point  that  bears  emphasizing  is  the  different  goals  of  the  tests. 
Basically,  a  NRT  is  designed  to  optimize  discriminabllity  between  all 
individuals  administered  the  test.  A  CRT  is  designed  to  maximize  discri- 
minability  around  the  cut-off  point  for  proficiency,  and,  technically,  is 
composed  only  of  items  necessary  for  identifying  proficiency  in  the  con¬ 
tent  domain  being  tested.  In  terms  of  test  item  difficulty,  NRTs  tend  to 
contain  a  range  of  item  difficulties;  CRT  items,  on  the  other  hand,  are 

viewed  as  homogeneous. 

Despite  different  test  development  strategies,  both  types  of  tests 
will  yield  a  distribution  of  response  scores  (e.g.,  number  or  percent 
correct).  However,  with  a  NRT  one  expects  to  obtain  a  more  normal  distri 
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bution  of  scores  while  with  a  CRT  one  expects  a  somewhat  skewed  and  peaked 
response  distribution  on,  e.g.,  an  end  of  course  exam.  It  must  be  stressed 
that  since  ability  is  a  continuous,  not  all-or-none,  variable,  test  scores 
will  always  reflect  a  variety  of  abilities.  Variations  in  scores  are  not 
simply  measurement  error  on  either  a  CRT  or  NRT.  Therefore,  scores  on 
either  type  of  test  may  be  given  a  norm- referenced  interpretation.  Just 
as  pass/fail  standards  are  set  on  both  NRT  and  CRT,  one  can  discuss  an 
individual’s  score  in  relation  to  all  other  scores  from  the  exam  with 
either  a  CRT  or  NRT. 

With  respect  to  the  selection  of  a  standard  setting  procedure,  any 
Ifrocedure  can  be  applied  to  either  type  (NRT  or  CRT)  of  test.  However, 
the  literature  does  imply  that  once  a  cut-off  score  or  standard  is  set  on 
a  CRT,  one  may  denote  individuals  whose  scores  fall  above  the  sUndard  as 
"masters’*  of  the  domain  covered  by  the  test.  Individuals  whose  scores 
fall  below  the  cut-off  are  designated  as  "non-masters"  of  the  subject 
matter.  In  fact,  one  specific  purpose  of  a  standard  on  a  CRT  is  to  iden¬ 
tify  an  individual  as  either  a  master  or  non-master  of  a  particular  skill 
domain  (though  not,  necessarily,  as  more  or  less  masterful  than  another 
individual).  Appropriate  labeling  of  individuals  scoring  above  or  below 
the  cut-off  point  on  an  NRT  is  less  clear.  This  may  be  attributed  to  the 
fact  that  since  NRT  items  are  sampled  statistically  (i.e.,  randomly  from  a 
pool)  rather  than  on  strict  content  domain  grounds,  it  is  less  clear  what 
an  individual  would  be  a  master  of,  except  the  items  on  the  test.  The 
Inference  from  the  test  items  to  the  domain  of  skill  is  weaker  for  a  NRT 

than  for  a  CRT. 
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The  report  by  Buck  (1977)  provides  a  good  discussion  of  concerns  and 
standard  setting  procedures  for  NRTs  and  CRTs.  By  way  of  summary,  Buck 
states  that  "a  test  is  not  inherently  norm-referenced  or  criterion-refer¬ 
enced.  It  is  the  manner  in  which  a  test  is  developed  and  interpreted  that 
determines  whether  it  is  to  be  classified  as  norm-  or  criterion-ref¬ 
erenced.  It  is  conceivable  that  a  test  could  be  either  norm-  or  crite¬ 
rion-referenced  or  both  depending  on  the  way  in  which  it  is  developed, 
used  and  interpreted  Ip.  15]".  Indeed,  the  Army's  Project  A  measures 
encompass  both  NRT  and  CRT  aspects.  Due  to  the  scope  and  purpose  of  the 
project,  the  development  of  the  criterion  (job)  measures  was  based  upon 
careful,  comprehensive  identification  of  the  criterion  domain  followed  by 
non-random  sampling  of  tasks  within  the  domain  and  construction  of  test 
items  in  such  a  way  as  to  optimize  discrimination  of  ability  levels  among 

individuals^. 

Modes  Of  measurement.  By  and  large,  the  published  literature  on  sUn- 
dard  setting  deals  with  paper  and  pencil,  multiple  choice  (recognition) 
tests.  Applications  of  standard  setting  procedures  to,  for  example, 
hands-on,  rating,  or  interview  assessment  procedures  are  not  represented 
in  the  literature  (cf.  Shikiar,  et  al  1985).  This  is  not  to  say,  however, 
that  the  existing  procedures  or  the  principles  they  embody  cannot  be  made 
to  accommodate  different  testing  modes.  Standard  setting  procedures  can 
also  be  augmented  to  encompass  the  practice  of  using  multiple  tests  and 
multiple  test  modes  for  criterion  (job)  performance  measurement.  Alterna 


hlVl  more  detailed  description  of  the  1985r*‘ 

CH  Campbell,  R.C.,  Rumsey,  M.G.,  and  Edwards,  D.C.  (1985). 
and’ field-test  of  task-based  MOS-specific  criterion  measure 
VA:  US  Army  Research  Institute,  in  press. 
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Development 
Alexandria, 
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tive  approaches  are  described  in  Buck  (1977). 

Selecting  and  Applying  SUndard  Setting  Procedures;  General  Consid^- 
tions. 

Although  the  literature  does  not  offer  specific  guidance  on  selecting 
one  procedure  over  another  for  different  situations,  general  recommenda¬ 
tions  or  areas  for  consideration  can  be  identified.  Specific  areas  will 
be  addressed  in  the  following  paragraphs:  Acceptance  of  standards  and 
modifying  standards. 

Acceptance  of  standards.  One  important  consideration  is  the  selection 
of  the  judges  (standard  setters).  If  representation  of  the  end-users  is 
included  in  the  standard  setting  process,  the  likelihood  that  the  resul¬ 
tant  standards  will  reflect  the  Interests,  concerns,  and  needs  of  the 
users  is  increased.  In  the  context  of  selection  and  classification  in  the 
military,  it  would  be  prudent  to  include  individuals  from  the  personnel, 
training,  policy  and  field  communities  on  standard  setting  panels.  In¬ 
volving  several  judges  or  groups  of  judges  in  the  standard  setting  process 
may  help  to  promote  confidence  in  and  acceptance  of  the  standards.  When 
independent  groups  of  judges  employ  the  same  sUndard  setting  procedure 
and  arrive  at  similar  standards  (cf.  Andrew  and  Hecht,  1976),  confidence 

in  the  standard  will  be  increased. 

Another  consideration  for  increasing  acceptance  of  standards  is  re¬ 
lated  to  both  the  judges  involved  and  the  procedure  selected.  A  procedure 
that  seems  convoluted  to  the  judges  or  asks  them  to  make  decisions  they  do 
not  feel  qualified  or  knowledgeable  enough  to  make  is  unsatisfactory. 
However,  the  very  same  procedure  presented  to  a  different  group  of  judges 
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may  meet  with  a  more  satisfactory  response.  Any  procedure  that  causes 
judges  to  feel  uneasy  is  not  likely  to  result  in  a  cut-off  score  that 
users  will  feel  confident  in  implementing;  the  validity  of  the  sUndard 
will  be  called  into  question.  This  is  not  to  say  that  judges  should  be 
selected  to  "fit"  the  procedure.  Rather,  it  is  recommended  that  a  proce¬ 
dure  should  be  selected  to  "fit"  the  standard  setters.  Every  effort 
should  be  made  to  ensure  that  that  judges  find  the  procedure  credible  and 

easy  to  apply. 

Several  sources  have  suggested  providing  judges  with  normative  data  so 
that  their  expectations  of  performance  will  not  be  unreasonable  (e.g., 
Livingston  and  Zelky,  1983;  Shepard,  1976).  Incorporation  of  normative 
data  is  likely  to  result  in  similar  sundards  across  judges  which,  in 
turn,  is  likely  to  improve  confidence  in  the  selected  standard.  Further, 
it  should  be  noted  that  Jaeger  et  al.  (1984)  have  found  that  iterative 
applications  of  a  given  sUndard  setting  procedure  result  in  reduced 
variability  across  judges.  Iterative  applicfations,  however,  did  not  af¬ 
fect  the  mean  recommended  standard.  The  reduction  in  variability,  i.e., 
better  agreement  among  judges,  is  also  likely  to  Increase  the  confidence 
of  the  judges  in  the  resultant  standards. 

Modifying  standards.  .The  preceeding  discussion  has  concentrated  on 
ways  to  maximize  confidence  in  standards  derived  by  any  given  procedure. 

It  is  important  to  ensure  not  only  that  the  judges  themselves  are  confi¬ 
dent  with  the  standard  but  also  that  the  end-users  and  individuals  dir¬ 
ectly  affected  by  the  use  of  the  standards  accept  the  results.  Therefore, 
In  addition  to  the  above  considerations,  institutional  requirements  and 
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values  must  also  be  taken  into  account.  No  matter  how  confident  judges 
may  be  in  their  decisions,  if  the  standards  appear  too  high  or  too  low 
from  an  institutional  perspective  the  standards  will  not  be  acceptable. 

It  was  suted  earlier  that  some  procedures  consistently  yield  higher 
standards  than  others.  This  means  that  some  procedures  will  result  in 
relatively  high  standards  that  are  likely  to  produce  false  negative  deci¬ 
sions,  i.e.,  individuals  will  be  classified  as  not  minimally  acceptable 
when,  in  fact,  they  would  have  been  able  to  perform  at  a  level  acceptable 
to  the  organization.  Conversely,  procedures  resulting  in  lower  standards 
are  more  likely  to  produce  some  amount  of  false  positive  decisions.  At  an 
organizational  level,  then,  consideration  may  be  given  as  to  whether  false 
negative  or  false  positive  decisions  are  more  serious  or  costly  to  the 
organization, 

.  Decision  theoretic  and  utility  analyses  can  serve  as  a  tool  to  "fine 
tune"  a  standard  set  by  panels  of  judges.  Decision  theory  is  not  a  stan¬ 
dard  setting  procedure;  it  is  a  technique  for  reducing  the  effects  of 
measurement  and  sampling  error  (van  der  Linden,  1980).  The  goal  of  util¬ 
ity  analysis  is  "to  match  the  test  dichotomy  to  the  criterion  dichotomy  to 
ensure  that  the  smallest  number  of  classification  errors  will  be  made" 
(Shepard,  1983).  One  form  of  utility  analysis  is  to  determine  the  rela¬ 
tive  cost  of  one  kind  of  error  (false  negative)  against  the  cost  of  an¬ 
other  kind  of  error  (false  positive).  This  may  be  a  difficult,  complex 
approach  to  apply  especially  with  respect  to  deciding  which  cost  factors 
should  be  used  (e.g.,  cost  of  training,  equipment  loss,  dollar  value  of 
performance) . 


10 


Eaton  et  al.  (1985)^,  have  presented  utility  estimation  techniques 
developed  to  be  easier  to  apply  in  situations  where  "managers  are  more 
accustomed  to  considering  the  relative  productivity  of  employees  or  crews 
than  the  costs  of  producing  given  levels  of  output  ...[or]...  where  em¬ 
ployees  operate  very  complex,  expensive  equipment  and/or  are  focal  to  the 
productivity  of  a  costly  system  [p.  29]".  The  strategies  presented  by 
Eaton  et  al.  consider  changes  in  the  number  and  performance  level  of  sys¬ 
tem  units  for  increased  aggregate  performance.  As  noted  by  the  authors, 
these  techniques  still  do  not  provide  for  easy  linkage  of  performance 
quality  to  a  single  quantitative  scale.  The  "linkage"  maybe  require  com¬ 
plex  judgements  regarding  the  utility  equivalence  of  different  performance 
levels  for  different  situations  or  groups  of  individuals. 

Within  Project  A,  attempts  are  being  made  to  scale  the  value  of  dif¬ 
ferent  levels  of  performance.  Utility  scaling  workshops  will  be  conducted 
with  military  personnel.  Their  task  will  be  to  scale  different  perform¬ 
ance  levels  of  various  Army  occupations  using  the  50th  percentile  perform¬ 
ance  of  the  infantryman  occupational  speciality  (IIB)  as  a  baseline.  It 
is  conceivable  that  the  resultant  scale  value  for  the  utility  of  individu¬ 
als  performing  at  the  90th  percentile  in  some  occupations  will  be  lower 
than  the  scaled  utility  of  the  50th  percentile  IIB. 

Once  the  utility  of  performance  levels  is  scaled  onto  a  single  dimen¬ 
sion,  information  obtained  from  the  scale  may  be  used  to  modify  test  per¬ 
formance  and/or  entrance  standards  for  different  occupations  in  order  to 


^Eaton,  H.K. , 
estimating  the 
27-40. 


Wing,  H.  and  Mitchell,  K.  (1985).  Alternative  methods  of 
dollar  value  of  performance.  Personnel  Psychology ,  _££» 
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optimize  selection  and  classification  decisions.  If,  as  expected,  a  given 
performance  level  (standard)  does  not  have  the  same  utility  across  occu¬ 
pational  specialties,  then  the  classification  standards  for  each  occupa¬ 
tion  may  be  modified  In  order  to  optimize  the  overall  utility  of  the  total 
enlisted  force. 

Conclusions 

This  review  has  started  from  the  premise  that  tests  on  which  standard 
setting  procedures  are  applied  have  already  been  determined  to  be  psycho- 
metrically  sound.  This  is  to  say  that  the  test  must  be  valid,  reliable, 
and  follow  the  guidelines  of  the  American  Psychological  Association^  for 
test  development  practices.  Once  a  test  has  been  appropriately  developed, 
a  test  performance  standard  (cut  score)  can  be  set. 

Each  standard  setting  procedure  should  be  applied  judiciously  " care 
should  be  taken  so  that  the  mathematics  involved  in  some  of  the  procedures 
do  not  create  a  false  sense  of  rigor.  Every  standard  setting  procedure  is 
based  on  judgement.  It  is  the  responsibility  of  the  developers,  users, 
and  overseers  of  the  standard  setting  process  to  ensure  that  the  judge¬ 
ments  are  sound,  appropriate  to  and  supportive  of  the  goals  and  values  of 
the  organization  or  community  served  by  the  standards  (cf.  Hofstee,  1983). 
Further,  it  is  incumbent  on  the  responsible  parties  to  evaluate  and  re¬ 
evaluate  the  standards  in  terms  of  the  impact  the  standards  have  on  the 
organization.  The  basic  objective  of  any  sundard  is  to  help  attain  the 

^American  Psychological  Association.  (1985).  Standards  for  educational  and 
psychological  testing.  Washington,  DC:  Author. 

American  Psychological  Association,  Division  of  Industrial-Organization 
Psychology.  (1980).  Principles  for  the  validation  and  use  of  personnel 
selection  procedures.  (Second  edition)  Berkeley,  CA;  Author. 
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critical  goals  and  requirements  of  a  given  institution.  Jobs,  for  example, 
change  as  do  the  needs  of  the  organization.  Accordingly,  measures  of  job 
performance  and  standards  based  on  those  measures  must  be  regularly  ap¬ 
praised  and  modified  as  needed  to  ensure  that  the  values  of  the  organiza¬ 
tion  are  being  met. 

The  concluding  point  of  this  review  is  that  there  cannot  be  one  and 
only  one  correct  sundard.  The  notion  that  one  correct  standard  can  be 
determined  for  a  given  situation  is  logically  inconsistent  with  the  fact 
that  performance  or  ability  exists  as  a  continuous  variable.  Any  stan¬ 
dard,  no  matter  how  it  is  derived,  imposes  an  artifical  dichotomy  (e.g., 
pass  vs  fail,  master  vs  non-master,  etc).  This  not  to  suggest  that  stan¬ 
dards  should  be  eliminated  or  avoided.  Standards  do  serve  as  useful  tools 
in  the  selection  and  classification  processes.  Rather,  the  standard  set¬ 
ting  process  cannot  end  with  the  determination  of  a  particular  standard. 
There  is  a  need  to  continue  evaluating  the  standard  to  ensure  that  the 
number  and  cost  of  the  inevitable  decision  errors  produced  by  that  stan¬ 
dard  are  minimized. 
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APPENDIX  A 


logical  Measurement,  36,  45-50. 

Abstract 

Two  standard  setting  procedures  were  employed  by  two  ^ 

judges  to  set  pass-fail  levels  for  comparable  samples 
tioLlly  administered  examination.  These  . 

designed  to  set  standards  In  relation  to  the  ralnimally  qualified 
examinee.  The  study  was  undertaken  to  determine  whether  similar 

Tunaari.  woald  ba  set  for  tha  aama  f!” 

determined  by  different  groups  of  judges,  and  whether  the  two 
procedures  employed  would  result  In  similar  standards  ^®^  P 
rable  samples  of  test  content.  In  addition,  the  extent  to  which 
group  consensus  judgments  might  differ  from  Individual 
lls  hso  investigated.  The  results  suggest  that  different  groups 
of  ludges  do  set  similar  examination  standards  when  using  the 
same  procedure,  and  that  the  average  of  Individual  judgments  does 
not  differ  significantly  from  group  consensus  judgments.  Slg 
niflcant  differences  were  found,  however,  between  the  sUndard 
set  by  the  two  procedures  employed.  This  finding  was  observ 
for  both  groups.  The  nature  of  these  differences  Is  described, 
and  their  Implications  for  setting  examination  standards  are 
discussed.  (Author) 


Aneoff.  W.H.  (1971).  Scales,  norms,  and  equivalent  scores. 
Thorndike  (Id-  )’,  F.duca\lonal  Measurement  (pp.  508-600).  Washington. 
American  Council  on  Education. 


In  R.L. 
DC: 


-  Pg.  514-515:  description  of  procedure; 

Systematic  procedure  for  deciding  on  minimum  raw  scores  for 
passing  and  honors  -  think  of  ••minimally  acceptable  person 

-  go  through  test  Item  by  Item; 

-  could  such  a  person  answer  correctly  the  Item  under 
consideration:  correct  score  1 

Incorrect  score  0 

-  sum  of  scores  »  raw  score  (cut  off]  of  minimally  ac¬ 
ceptable  person 

-  have  some  number  of  Independent  judges  decide  by  con¬ 
sensus  without  actually  administering  the  tests 

-  results  could  later  be  compared  with  numbers  and 
percent  of  examinees  who  actually  earned  the  passing 
grades  (validity  studies:  verify  appropriateness  of  the 
Initial  cutting  scores  or  correct  them  If  necessary] 

-  or  ask  each  judge  to  state  the  probability  that  the 
minimum  acceptable  person  would  answer  each  Item 
rectly;  the  sum  of  the  probabilities  would  represent  the 
minimum  acceptable  score 


-  Pg.  531... suggestion  of  applying  cut  score  procedure  to  Army. 


Block,  J.H.  (1978).  Standards  and  criteria:  A  response.  Journal 
of  Educational  Measurement,  15,  291-295. 

-  Responds  to  Glass  (1978)  paper 

-  Argues  that  standard-setting  techniques  are  not  as  arbitrary  as 
Glass  suggests 

-  Suggests  developing  new  and  better  technique . promote  broad 

basfed  humanistic  procedure. 


T  c  (iQ77'i.  Guide  to  the  setting  of  appropriate  cutting 
.rcr.f  Tor  !;;i;rrn  the  concerns  and 'iT^cedures  (Tech- 

nlcal  hLorandum  77-4).  VaiFlngton.  DC:  Personnel  Research  and  Develop¬ 
ment  Center  United  States  Civil  Service  Commission. 


-  Presents  a  review  and  summary  of  methods  for  establishing  cut 
scores 


-  'Discusses  Issues  as  they  apply  to  norm  and  criterion  reference 
tests 

-  Pg.  9:  summary  of  issues  and  models  of  test  fairness 

-  Pg.  13-15:  summary  -  cutting  scores  for  NRTs 

-  Pe.  20-21:  summary  -  cutting  scores  for  CRTs 

(10  "models’’...  methodology  different  ...  but  no  em¬ 
pirical  or  theoretical  basis  for  selecting  one  over 
another) 


-  Pe  13:  “The  test  developer  must  realize  that  the  process  of 
setting  a  cutting  score  cannot  be  totally  analytic,  as  it  is  im 
possible  to  assume  a  purely  objective  attitude". 


Buck  L  S  (1975).  Ose  of  criterion-referenced  tests  In  personnel 
selectioni  A  summary  statUs^epo^Teclmlcal  Memorandum  75-6). 
tof,  DC:  Personnel  KeTearch  and  Development  Center  United  States  Civil 

Service  Commission. 


-  Discusses  validity  of  CRTs  and  measures  of  reliability  (actually 
does  little  more  than  reference  papers  dealing  with  the  topics; 


-  See  pg.  21  ...  measures  of  reliability. 

pg.  23  ...  validity  . 

pg.  26  note  ...  rel labll Ity/correlatlon  estimate 


-  Provides  28-page  annotated  bibliography 


Burton,  N.W.  (1978).  Societal  standards.  Journal  of  Educational 
Heasuremen t ,  15,  263-272. 

-  Pg.  26A  -  definition  of  criterion  -  differences  In  emphasis: 

1)  criterion  [variable]  -  trait  to  be  measured  (traditional 
definition) 

2)  —  specification  of  minimum  levels  of 

performance  (Glaser  and  CRTs) 

-  Pg.  266  -  3  types  of  methods;  1)  standards  based  on  theories 

(learning  hierarchies) 

2)  standards  based  on  expert 
con sen sus 

3)  standards  based  on  practical 
necessities  (minimal  compe¬ 
tencies  for  real-life) 


Chuang,  D.T.,  Chen,  J.J.,  &  Novlck,  M.R. 
the  use  of  cut-scores  for  personnel  decisions 
tics,  6,  129-152. 


(1981).  Theory  and  practice  for 
Journal  of  Education  Statls- 


-  Mathematical  model 

-  Optimize  utility  ...  final  cut-score  set  by  utility  function... 

•••  assumes  some  cut  score  has  already  been 
determined.  •  • 


-  Does  not  specify  how  cut  scores  are  set 


Du  Bo lS|  P»H«y 
proficiency  tests. 


Teel,  K.S..  &  Petersen,  R.L.  (1954).  On  the  validity  of 
Educational  and  Psychological  HeasuremerU,  JU,  605-616. 


-  A  proficiency  test  is  considered  valid  if  it 
tween  the  proficient  and  non-proficient  in  a  given 
an  aptitude  test  may  have  an  Indefinite  number  of  validities,  de- 
pen(31ng  on  the  criteria  which  it  predicts  with  varying  degrees  o 
success,  a  valid  proficiency  test  must  measure  what  It  purports  to 
measure.  No  other  concept  of  validity  is  applicable,  the  only 
variation  possible  is  in  the  method  used  in  arriving  at  the  esti¬ 
mate  of  validity"  (pg.  605) 


-  Coverage  and  discrimination  power  are  Independent  dimensions  of 
a  proficiency  test 


-  Difficulty  analysis  -  item  has  p-value  of  .50  (passed  by  50%  of 
the  croup)...  Its  SD  is  at  maximum  and  makes  maximum  numbers  of 
discriminations...  A  range  of  item  difficulties,  from  very  «a®y 
to  very  hard  with  a  mean  at  about  .50  is  optimal  for  differentiat¬ 
ing  within  a  given  population 


-  Types  of  validity: 

-  Validity  by  "Direct  Judgment"  —  uses  SMEs 

-  Work  sample  validity  —  correlation  with  work  sample  that 
is  representative  and  meaningfully  measures  the  skill 

-  Class  validity  —  e.g.  high  vs.  medium  vs.  low  proficiency 

-  Curricular  validity  —  untrained  vs.  trained 


Eastman,  R.F.  (1981).  Supervisor  ratings  as  criteria  for  Skill 
Qualification  Tests.  In  S.F.  Bolin  (Chair),  Panel  on  skill  quaiyication 
usting:  An  .vol.lng  syst..  (pp.  1356-1366). 

Cpnference^'of  the  Military  Testing  Association.  [DflC,  ADP  001400] 


-  Correlation  between  supervisor  ratings  of  overall  job  perform¬ 
ance  and  SQT  scores  for  67N  (r=.74) 


-  Used  ratings  as  criteria  to  determine  optimum  cut  score  for 
performers  vs*  non-performers 


ratings:  -  competent/not  competent/don't  know  .  u 

-  cross  out  name  If  you  don’t  supervise  the  Individ¬ 
ual 

-  use  a  "+*■  sign  to  indicate  one  of  the  best  soldiers 
and  a  sign  to  indicate  one  of  the  poorest  soldiers 


-  Each  soldier  rated  by  3-6  supervisors 

-  Plot  (tabulate)  distribution  of  performers  (rating  >  3)  by 
score 

-  Findings:  SQT  cut-score  could  be  lowered  and  be  more  consistent 
with  perceptions  of  supervisory  personnel;  it  may  be  better  to  have 
supervisor  rank-order  soldiers  instead  of  having  to  designate  a 
soldier  as  a  non-performer. 


Glass,  G.V.  (1978).  Standards  and  criteria.  Journal  of  Educatiqn_al 
Measurement,  15,  237-262. 


-  Pe.243:  Criterion:  the  definition  has  been  corrupted;  originally 
referred  to  a  criterion-referenced  test  ...  meaning  a  scale  of  be¬ 
havior  linked  to  a  test  scale  ...  now  criterion  uken  to  be 
synonymous  with  "standard"  or  "cut  score" 

-  Glass  expresses  strong  concern  regarding  th^rbitrary  nature  of 
setting  a  standard  and  the  notions  of  a  standard 

-  Six  classes  of  methods  for  establishing  cut-offs. 

1.  Performance  of  others  -  reference  parameters  of  existing^ 
popToiTTolTTf  examine'ir-  e.g.,  median  score,  50th  percentUe 
...  essentially  normative...  not  "behaviorally  informative  ; 
criterion-reference  test  theorists  would  find  this  approach  to 

be  an  inappropriate  method.  .  _ 

2.  "rmintlng  Backwards  from  100%"  -  given  the  nature  of  criter 
ion-references  test  (objectivesT. • .would  expect  perfect 
scores. ..but  allowances  must  be  made  for,  e.g.,  measurement 
error  and  clerical  mistakes. . .but  how  much??... the  method  is 
highly  judgmental  and  too  vague 

3.  Bootstrapping  on  other  criterion  scor^  -  use  other  deter- 

mln^ti^^^Fl^^'competence  to  select  a  group  then  match  the  group 
against  the  score  distribution  of  some  other  test. . .problems 
1)  the  2  tests  must  be  correlated,  but  It  will  never  b(  per¬ 
fect,  therefore,  you  will  make,  e.g.,  false  positive  :  .alse 

negative  decisions,  and  you  still  have  the  problem  of  - 

was  the  standard  set  on  the  first  test. . .circularity  problem 

4.  Judging  minimal  competence  -  study  a  test  and  determine  the 

required  score  for  a  minltiniy  competent  individual  (cf.  Nedel- 
sky;  Ebel)...see  pg.  246  for  Nedelsky  method;  pg  247  for  Ebel 
method;  pg.  248  for  Angoff  method;  problems:  1)  consistency 

and  reliability  of  judges;  2)  logical  psychological  status  of 
concept  of  minimal  competence. 

5.  Decision  theoretic  approach^  -  cutoff  on  an  external  crite- 
rion  assumed  as  a  "given" ..  .va^  score  on  the  criterion- 
referenced  test  to  say,  minimize  false  negatives . ..approach 
simply  postpones  decision  regarding  the  setting  of  a  cut  off... 

still  "arbitrary"  ,  , 

6.  "Operations  Research"  Methods  -  based  on  OR  approach  of 
maximizing  a  valued  commodity  by  finding  an  optimum  point  on  a 

mathematicalAurve  or  graph -  must  have  a  non-mono tonic 

curve. ..could  have  composite  with  a  second  valued  outcome,  but 
then  have  the  problem  of  how  to  weight  the  composite... or  look 
for  the  point  of  diminishing  returns. . .(no  further  gain)  how 
do  you  decide  non-arbitrarlly 

-  Pe  258...  standard-setting  procedure  may  involve  more  precision  than  the 
UsVltteif  ta“..  no  nattnt  .hat  procedure  need,  there  1,  .till  the  ale- 
ment  of  the  arbitrary 

-  Glass  favors  a  comparative  approach,  e.g.,  improvement  Per¬ 

formance)  but  you  still  have  the  questions  re:  how  much  change  is  good/ 
Lfficient...hL  much  loss  before  action  should  be  taken... 

as  with  criterion  score,  but  he  claims  one  has  still  gaine  y 

consensus  even  if  all  problems  were  not  solved. 


Hambleton,  R.K.  (1978).  On  the  use  of 
referenced  tests  In  instructional  settings, 
urement,  15|  277-290. 


cut-off  scores  with  crlterlon- 
Journal  of  E<3ucattonal  Meas- 


-  Validity  of  cut  score  depends  on  how  accurately  it  separates 
examinees  Into  mastery  states. . .usually  the  criterion  Is  some 
external  measure  of  performance  or  Instructed  vs.  non-lnstructed 

groups 


-  Methods  are  based  on  the  consideration  of  Item  content,  educa¬ 
tional  consequences,  psychological  and  financial  costs,  per  orm 
ance  of  others,  errors  due  to  guessing  and  Item  sampling. . .aii 
arbitrary 


-  Against  Glass'  (1978)  recommendation  of  using  change  scores 


Hambleton,  R.K.  &  Eignor,  D.R.  (1980).  Competency  f 

validation,  and  standard  setting.  In  R.M,  Jaeger  &  C.K.  Tittle  (  .), 

Minimum  competency  achievement  testing; — Motives, 

^Si^uences  (pp.  367-396).  Berkeley,  CA:  McCutchan  Publishing  Corpora 
tion. 

-  Focuses  on  making  competency  judgment  for  individuals  -  not 

groups  (e.g _ program  evaluation) 

-  "A  minimum  competency  test  is  designed  to  determine  whether  an 
examinee  has  reached  a  prespecified  level  of  performance  necessary 
to  each  competency  being  measured" .. ."standard  [or  cutoff  sc 

or  "minimal  proficiency  level"]  is  a  point  on  a  test  score  scale 
which  is  used  to  separate  examinees  into  two  categories  ...mas¬ 
ter/nonmaster  -  a  standard  is  set  for  each  competency  measured  by  a 
test. . ."competency  tests  are  a  special  type  of  criterion-referenced 
test"  -  requires  "information  about  levels  of  individual  perform¬ 
ance  relative  to  well-defined  content  domains  (referred  to  as 
"domain  specifications)" 


-  4  important  topics:  1)  improved  guidelines  for  preparing  domain 

specifics  tlons 

2)  guidelines  for  evaluating  competency 
tests  and  test  manuals 

3)  research  on  the  relationship  among  test 
length,  test  score  reliability,  and  test 
score  validity 

4)  consideration  of  issues  and  methods  for 
determining  standards,  as  well  as  guide¬ 
lines  for  implementing  each  method 


-  Goes  through  12-step  model  for  developing  and  validating  compe¬ 
tency  tests 

_  pg,  377:  test  length  formula  for  criterion  reference  tests,  (vs. 
Spearman-Brown  for  norm-referenced  tests) 

-  Continuum  vs.  state  models  (all-or-none) ;  in  the  latter,  test 
true-score  performance  is  viewed  as  all-or-none,  true-score  stan¬ 
dard  is  set  at  100%,  after  consideration  of  measurement  error  the 
observed-score  standard  is  set  at  a  value  less  than  100%... use 
normative  information  as  an  aid  in  making  decisions  (in  case  expe¬ 
rience  of  judges  may  have  been  with  unusual  students) 


♦See  pg 
values 
pg.  386 
pg.  392 


383-384. . .different  models'  use  of  (need  for)  utility 


-  comparison  of  standard  setting  procedures 

-  for  empirical  methodology  and  the  need  for  external 

criterion  measures  (see  refs.) 
pg  3  -  latent  trait  models. . .feasibility  with  competency  tests??, 
equating  scores  from  one  form  of  competency  test  to  another. 
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CHAPTER  3 


ARMY  RESEARCH  TO  LINK  STANDARDS  FOR  ENLISTMENT  TO 
ON-THE-JOB  PERFORMANCE 

CURRENT  STATUS  AND  ACCOMPLISHMENTS 


The  Army  Research  Institute  is  currently  engaged  in  a  large- 
scale,  multi-year  project  to  improve  the  Army  selection  and 
classification  system  and,  thereby,  increase  the  overall  effecti¬ 
veness  of  the  force.  The  goal  of  the  Army’s  program  for  increas¬ 
ing  the  efficiency  of  enlisted  personnel  selection  and  utiliza¬ 
tion  is  to  enable  the  Army  to  meet  its  peacetime  and  mobilization 
missions  through  improved  matching  of  individuals  to  Military 
Occupational  Specialties  (MOS).  The  research  is  aimed  at 
developing  comprehensive  selection  and  classification  procedures 
to  validly  predict  performance  in  Army  training  and  occupational 
specialties. 

A  complete  description  of  the  Army’s  research  program  and 
accomplishments  have  been  published  separately  by  the  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences  in  the 
annual  reports  for  Project  A,  "Improving  the  Selection,  Classifi¬ 
cation  and  Utilization  of  Army  Enlisted  Personnel"  (1983,  1984, 
1985,  1986).  A  detailed  description  of  the  project  also  appears 
in  previous  Annual  Reports  to  Congress  on  Joint-Service  Efforts 
to  Link  Standards  of  Enlistment  to  On- the- Job  Performance  (1983, 
1984,  1985,  1986). 

The  Army’s  research  focuses  on  19  MOS.  The  MOS  were  selected 
to  be  representative  of  the  Army  and  include  all  operational 
ASVAB  aptitude  area  composites.  Blacks,  Whites,  Hispanics, 
males,  and  females  are  present  in  these  MOS  in  the  same  propor¬ 
tions  as  in  total  accessions.  These  MOS  represent  44Z  of  annual 
Army  enlistments. 

A  ntunber  of  performance  measures,  including  measures  of 
training  success,  service-wide  performance,  and  MOS-specific 
hands-on  performance,  were  developed  for  these  MOS.  For  reasons 
of  cost  efficiency,  not  all  measures  were  developed  for  all  19 
MOS.  All  project  criterion  measures  were  developed  for  the 
following  MOS  and  serve  as  the  focus  of  this  report; 

1.  IIB  -  Infantryman 

2.  13B  -  Cannon  Crewman 

3.  19E  -  Tank  Crewman 

4.  31C  -  Radio  Teletype  Operator 

5.  63B  -  Light  Wheel  Vehicle/Power  Generation  Mechanic 

6.  64C  -  Motor  Transport  Operator 

7.  71L  -  Administrative  Specialist 

8.  91A  -  Medical  Specialist 

9.  95B  -  Military  Police 


Measures  of  training  success  and  service-wide  performance  were 
developed  for  the  following  specialties; 

1.  12B  -  Combat  Engineer 

2.  16S  -  MANPADS  Crewman 

3.  27E  -  TOW/Dragon  Repairman 

4.  51B  -  Carpentry /Masonry  Specialist 

5.  54E  -  Chemical  Operations  Specialist 

6.  55B  -  Ammunition  Specialist 

7.  67N  -  Utility  Helicopter  Repairman 

8.  76W  -  Petroleum  Supply  Specialist 

9.  76y  -  Supply  Specialist 

10.  94B  -  Food  Service  Specialist 

A  complete  report  of  accomplishments  and  current  status  of 
the  Army’s  research  to  link  enlistment  standards  to  job  perform¬ 
ance  has  been  published  by  the  Army  Research  Institute  (1983, 
1984,  1985,  1986)  and  is  available  for  distribution.  What 

follows  includes  a  summary  of  the  Army’s  accomplishments  in  the 
area  of  job  performance  measurement. 

As  reported  last  year  in  the  Annual  Report  to  Congress  the 
Army  has  completed  the  concurrent  validation  phase,  i.e.,  joint 
administration  of  predictor  and  criterion  measures,  of  its 
project.  The  research  not  only  established  the  feasibility  of 
measuring  job  performance  but  also  identified  five  performance 
constructs,  described  later  in  this  chapter,  thereby  providing 
support  for  the  notion  that  job  performance  is  multi-dimensional. 
In  addition,  the  research  established  that  there  is  a  positive 
relationship  between  ASVAB  and  job  performance  during  the  first 
term  of  service.  More  specifically,  the  higher  an  applicant’s 
ASVAB  score  the  better  he  or  she  performs  the  technical  or  "can- 
do"  aspects  of  the  job.  The  correlations  ranged  from  .23  to  .68. 
The  will-do  or  motivational  factors  are  not  well-predicted  by 
ASVAB,  but  the  Army’s  new  predictor  measures  are  expected  to 
improve  the  prediction  of  the  non-cognitive  job  performance 
dimensions. 

The  Army  is  currently  engaged  in  the  longitudinal  validation 
phase  of  its  project.  In  the  past  year,  fifty- three  thousand 
Army  entrants  were  administered  the  Army  project’s  predictor 
battery  at  eight  reception  battalions.  Collection  of  training 
(school)  knowledge  measures,  administered  to  soldiers  in  the 
sample  at  the  end  of  Advanced  Individual  Training  (AIT),  has 
begun  and  will  end  in  May  of  1988.  Administration  of  the  job 
performance  measures  is  scheduled  to  be  conducted  between  July 
and  November  1988.  The  measures  are  currently  being  revised  and 
pilot  tested;  they  will  be  field  tested  between  January  -  March. 

Implementation  of  the  project’s  findings  currently  includes 
three  efforts.  First,  with  respect  to  the  use  of  ASVAB, 
modifications  have  been  made  to  the  aptitute  area  composites  to 
improve  can-do  assignment  efficiency.  Forty-four  MOS  are  in  the 
process  of  being  reassigned  to  different  entry  composites  in 


order  to:  increase  validity,  increase  consistency  of  composite 
use  within  a  career  management  field,  and  increase  the  accuracy 
of  prediction  for  women  and  blacks.  The  second  major  implement¬ 
ation  effort  involves  one  of  the  new  predictors.  Assessment  of 
Background  and  Life  Experiences  (ABLE) ,  a  temperament/personal 
history  instrument.  The  Army  is  presently  planning  for  an 
Initial  Operational  Test  and  Evaluation  (lOT&E)  of  ABLE.  ABLE 
will  help  the  Army  to  predict  better  the  job  dimensions  of: 
Effort  and  Leadership,  Personal  Discipline,  and  Physical  Fitness 
and  Military  Bearing.  The  third  implementation  is  designed  to 
better  capitalize  on  individual  psychomotor  and  spatial 
attributes  in  MOS  which  require  them  for  improved  gunnery 
performance. 


RESEARCH  DESIGN 


Types  of  Measures  Developed 

Hands-on  performance  measures,  job  knowledge  tests,  and 
performance  rating  scales  were  developed  for  training  success, 
service-wide  performance,  and  MOS-speclfic  performance  for  the 
MOS  listed  previously.  The  Army’s  rationale  for  the  development 
of  multiple  measures  of  job  performance  is  based  upon  the 
knowledge  that  a  soldier’s  job  is  multi-faceted  and  there  are 
different  aspects  of  job  performance.  Therefore,  the  Army’s 
research  project  has  developed  different  kinds  of  tests  to  assess 
these  different  aspects  of  job  performance  and  thereby  obtain 
information  about  the  domain  of  job  performance  behaviors.  A 
more  complete  description  of  these  measures  appears  in  the 
Project  A  Annual  Reports  (1983,  1984,  1985,  1986)  and  in  previous 
Annual  Reports  to  Congress  (1983,  1984,  1985,  1986). 

Data  Collection  Procedures 


Data  collection  began  with  the  briefing  of  local  military 
commanders,  examination  of  the  test  sites,  equipment  and  sup¬ 
plies,  training  of  test  administrators  and  scorers,  and  a  dry  run 
of  testing  procedures.  Test  site  managers  were  appointed  to 
supervise  the  actual  data  collections  and  were  responsible  for 
controlling  the  quality  and  flow  of  data  from  the  testing. 

The  concurrent  validation  data  collection  began  in  June  and 
ended  in  November  1985.  As  reported  in  the  Fourth  Annual  Report 
to  Congress,  data  were  collected  at  fourteen  different  sites. 
The  data  from  5200  soldiers  in  9  MOS,  who  were  tested  on  all  the 
project  criterion  measures,  were  entered  into  the  research  data 
base.  Data  from  an  additional  4000  soldiers  in  the  10  other  MOS, 
who  were  tested  only  with  the  measures  of  training  success  and 
service-wide  performance  were  also  entered.  Analyses  began  as 
soon  as  all  the  data  were  collected  and  entered  in  the  lon¬ 
gitudinal  research  data  base. 


RESULTS  OF  DATA  ANALYSES 


Common  Data  Analyses 

In  order  to  provide  comparable  data  across  services,  the 
Principal  Deputy  Assistant  Secretary  of  Defense  for  Force 
Management  and  Personnel  directed  the  following  analyses  be 
conducted  by  each  service.  The  results  of  the  analyses  are 
presented  below. 

Sample  Description.  The  analyses  are  based  on  data  collected 
during  the  Army’s  concurrent  validation  data  collection  in  1985. 
Soldiers  were  in  Skill  Level  One,  entry-level  positions  in  their 
respective  MOS  primarily  with  18-24  months  experience  in  the 
Army. 

Table  1  presents  descriptive  statistics  for  the  variables 
included  in  these  analyses.  It  should  be  noted  that  the  hands- 
on  performance  test  (HOPT)  scores  have  been  converted  to  standard 
T-scores  in  order  to  allow  comparison  across  MOS.  Time  in 
service  is  the  number  of  months,  averaged  across  soldiers,  of 
Army  service.  In  accordance  with  the  Army’s  sampling  plan,  the 
means  are  between  18  and  24  months,  with  small  variances 
(standard  deviations).  Average  AFQT  percentile  score  for  each 
MOS  sample  is  also  presented,  along  with  the  ntimber  of  soldiers 
classified  as  high  school  diploma  graduates  (HSD6)  or  non-high 
school  diploma  graduates  (NHSDG). 

Table  2  also  presents  descriptive  statistics  for  the  sample, 
this  time  broken  into  four  groups  for  time  in  service  and  two 
AFQT  groups,  upper  50th  percentile  (I-IIIA)  and  lower  50th 
percentile  (IIIB-IV).  Again,  given  the  design  of  the  Army’s 
project,  few  soldiers  were  expected  in  the  1-12,  25-36,  and  37+ 
Time  in  Service  groups. 

Reliability  Results.  Two  indices  of  reliability,  split-half 
and  coefficient  alpha,  for  HOPT  were  computed  and  are  presented 
in  Table  3.  Both  uncorrected  and  corrected  reliability  estimates 
are  included  along  with  the  standard  error  of  measurement.  The 
table  also  includes  the  number  of  tasks  in  the  HOPT  scores  for 
these  common  data  analyses. 

Hands-On  Performance  Test.  Aptitude.  Education,  and  Job 
Experience  Relationships.  Intercorrelations  among  the  measures 
are  presented  for  each  MOS.  Uncorrected  correlations  are 
depicted  in  Table  4.  The  correlations  in  Table  5  have  been 
corrected  for  range  restriction  using  a  univariate  correction 
procedure  as  agreed  upon  by  the  Joint  Service  Job  Performance 
Measurement  Working  Group  for  the  purposes  of  these  analyses. 

Service  -  Specific  Data  Analyses 


We  believe  the  most  appropriate  data  showing  the  link  between 


Table  1 


Descriptive  Statistics  for  HOPT,  Time  in  Service, 
Aptitude,  and  Educational  Attainment 


HOPT  Time  Educational 


MOS 

(Standard 

T-Score) 

in  Service 
(Months ) 

AFQT 

(Percentile) 

Attainment 

(N) 

Mean 

50 

20.4 

56.2 

423 

HSD6 

IIB 

SD 

10 

5.6 

21.8 

55 

NHSDG 

N 

491 

478 

491 

Mean 

50 

21.8 

47.0 

353 

HSD6 

13B 

SD 

10 

6.1 

19.7 

73 

NHSDG 

N 

434 

425 

464 

Mean 

50 

19.4 

53.9 

317 

HSDG 

19E 

SD 

10 

4.6 

21.5 

50 

NHSDG 

N 

378 

367 

394 

Mean 

50 

20.5 

58.9 

245 

HSDG 

31C 

SD 

10 

5.6 

19.2 

29 

NHSDG 

N 

275 

274 

289 

Mean 

50 

19.9 

49.7 

397 

HSDG 

63B 

SD 

10 

5.0 

19.3 

64 

NHSDG 

N 

443 

461 

478 

Mean 

50 

20.0 

43.1 

420 

HSDG 

6AC 

SD 

10 

4.6 

18.3 

66 

NHSDG 

N 

481 

486 

507 

Mean 

50 

22.3 

56.5 

401 

HSDG 

71L 

SD 

10 

4.5 

19.2 

7 

NHSDG 

N 

416 

408 

427 

Mean 

50 

20.6 

59.1 

310 

HSDG 

91A 

SD 

10 

5.4 

18.3 

68 

NHSDG 

N 

392 

378 

392 

Mean 

50 

21.3 

62.1 

525 

HSDG 

95B 

SD 

10 

5.0 

17.3 

44 

NHSDG 

N 

577 

569 

597 

Table  2 


Mean  HOPT  Scores  (Standardized  T-Scores)  by  Aptitude 
and  Job  Experience  Levels  for  9  Jobs 


Time  in 
Service 
(Months ) 

MOS 

IIB 

MOS 

13B 

MOS 

19E 

AFQT 

I-IIIA 

AFQT 

IIIB-IV 

AFQT 

I-IIIA 

AFQT 

IIIB-IV 

AFQT 

I-IIIA 

AFQT 

IIIB-IV 

Mean 

52.8 

45.2 

41.2 

46.7 

47.0 

37.6 

1-12 

SD 

9.0 

10.4 

11.1 

11.8 

8.5 

N 

6 

7 

1 

5 

7 

4 

Mean 

51.9 

48.0 

49.8 

48.1 

51.0 

47.7 

13-24 

SD 

9.8 

10.3 

10.4 

9.3 

9.6 

10.8 

N 

213 

155 

108 

158 

150 

134 

Mean 

50.8 

48.3 

53.6 

52.3 

55.4 

52.3 

25-36 

SD 

9.2 

9.5 

10.4 

9.8 

6.6 

9.2 

N 

41 

47 

38 

74 

25 

32 

Mean 

49.5 

41.8 

54.5 

53.8 

__ 

51.7 

37+ 

SD 

8.4 

15.2 

11.6 

7.6 

... 

N 

4 

5 

5 

6 

— 

1 

Mean 

51.7 

47.8 

50.8 

49.5 

51.4 

48.3 

Total 

SD 

9.6 

10.2 

10.5 

9.6 

9.5 

10.7 

Group 

N 

264 

214 

152 

243 

182 

171 

MOS 

31C 

MOS 

63B 

MOS 

64C 

Time  in 
Service 
(Months ) 

AFQT 

I-IIIA 

AFQT 

IIIB-IV 

AFQT 

I-IIIA 

AFQT 

IIIB-IV 

AFQT 

I-IIIA 

AFQT 

IIIB-IV 

Mean 

49.2 

46.0 

40.7 

50.6 

54-9 

52.6 

1-12  SD 

9.7 

8.8 

9.1 

3.5 

11.3 

8.3 

N 

8 

7 

6 

7 

5 

6 

Mean 

52.0 

47.2 

51.3 

48.9 

51.3 

48.6 

13-24  SD 

9.9 

9.2 

10.4 

9.5 

10.0 

9.8 

N 

140 

74 

156 

193 

96 

258 

Mean 

50.2 

46.2 

50.0 

51.9 

55.0 

50.1 

25-36  SD 

10.6 

12.9 

10.3 

10.1 

8.9 

10.7 

N 

16 

10 

15 

47 

32 

64 

Mean 

51.9 

42.5 

25.1 

57.9 

37+  SD 

16.7 

9.0 

7.7 

«... 

N 

2 

3 

1 

3 

— 

— 

Mean 

51.7 

46.8 

50.7 

49.6 

52.4 

49.0 

Total  SD 

10.0 

9.5 

10.6 

9.6 

9.9 

9.9 

Group  N 

166 

94 

178 

250 

133 

328 

Table  2  (Continued) 


Mean  HOPT  Scores  (Standardized  T-Scores)  by  Aptitude 
and  Job  Experience  Levels  for  9  Jobs 


MOS 

71L 

MOS  91A 

MOS  95B 

Time  in 
Service 
(Months) 

AFQT 

I-IIIA 

AFQT 

IIIB-IV 

AFQT  AFQT 

I-IIIA  IIIB-IV 

AFQT  AFQT 

I-IIIA  IIIB-IV 

Mean 

48.9 

53.5 

49.8 

47.3 

49.5 

47.5 

1-12 

SD 

15.4 

13.5 

11.1 

10.2 

9.0 

8.0 

N 

6 

4 

10 

4 

18 

6 

Mean 

53.2 

46.5 

51.3 

47.2 

50.6 

47.3 

13-24 

SD 

9.8 

9.4 

9.8 

10.0 

9.7 

10.5 

N 

179 

119 

195 

88 

290 

111 

Mean 

51.4 

47.8 

51.2 

48.1 

52.3 

49.7 

25-36 

SD 

8.8 

8.9 

9.8 

10.6 

10.1 

10.2 

N 

49 

38 

42 

36 

83 

41 

Mean 

48.5 

44.7 

57.0 

50.3 

48.0 

37+ 

SD 

3.8 

— 

3.2 

— 

— 

— 

N 

2 

1 

2 

1 

1 

— 

Mean 

52.7 

47.0 

51.3 

47.5 

50.9 

47.9 

Total 

SD 

9.7 

9.3 

9.8 

10.1 

9.7 

10.3 

Group 

N 

236 

162 

249 

129 

392 

158 

Table  3 


Reliability  Estimates  and  Standard  Errors  of  Measurement 
for  Job-Specific  Hands-on  Performance  Tests 


MOS 

Number  of 
Tasks 

Split- 

Half 

Coefficient 

Alpha 

sem“ 

IIB 

14 

ucoef 

.56 

.55 

6.71 

ccoef 

.57 

.57 

13B 

17 

ucoef . 

.76 

.70 

5.52 

ccoef . 

.76 

.70 

19E 

15 

ucoef. 

.57 

.56 

6.66 

ccoef. 

.59 

.57 

31C 

15 

ucoef. 

.81 

.69 

5.58 

ccoef. 

.83 

.72 

63B 

15 

ucoef. 

.47 

.40 

7.74 

ccoef. 

.47 

.40 

64C 

16 

ucoef. 

.66 

.60 

6.32 

ccoef. 

.68 

.63 

71L 

15 

ucoef. 

.73 

.68 

5.65 

ccoef. 

.76 

.72 

91A 

17 

ucoef. 

.59 

.72 

5.31 

ccoef. 

.62 

.74 

95B 

16 

ucoef. 

.63 

.52 

6.90 

ccoef. 

.65 

.55 

^Standard  Error  of  Measurement  calculated  with  the  Coefficient 
Alpha  reliability  estimates  and  HOPT  standard  T-Score  standard 
deviations . 

^ncorrected  reliability  estimate. 

*^Corrected  reliability  estimate. 


Table  4 

Uncorrected  Intercorrelations 


Intercorrelations  Between  Measures  for  MOS  IIB 


Measure 


Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

-.06 

.23 

-.12 

2  Time  in  Service 

-.06 

1.00 

-.04 

-.05 

3  AFQT  Percentile 

.23 

-.04 

1.00 

-.06 

4  Educational  Attainment 

-.12 

-.05 

-.06 

1.00 

Intercorrelations 

Between  Measures 

for  MOS  13B 

Measure 

Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

.18 

.11 

-.02 

2  Time  in  Service 

.18 

1.00 

-.01 

-.04 

3  AFQT  Percentile 

.11 

-.01 

1.00 

.15 

4  Educational  Attainment 

-.02 

-.04 

.15 

1.00 

Intercorrelations 

Between  Measures 

for  MOS  19E 

Measure 

Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

.26 

.22 

.08 

2  Time  in  Service 

.26 

1.00 

-.01 

.09 

3  AFQT  Percentile 

.22 

-.01 

1.00 

.01 

4  Educational  Attainment 

.08 

.09 

.01 

1.00 

Intercorrelations 

Between  Measures 

for  MOS  31C 

Measure 

Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

.01 

.32 

-.04 

2  Time  in  Service 

.01 

1.00 

.03 

.03 

3  AFQT  Percentile 

.32 

.03 

1.00 

.03 

4  Educational  Attainment 

-.04 

.03 

.03 

1.00 

Intercorrelations 

Between  Measures 

for  MOS  63B 

Measure 

Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

.07 

.07 

.05 

2  Time  in  Service 

.07 

1.00 

-.10 

-.13 

3  AFQT  Percentile 

.07 

-.10 

1.00 

.09 

4  Educational  Attainment 

.05 

-.13 

.09 

1.00 

Table  4  (Continued) 
Uncorrected  Intercorrelations 


Intercorrelations  Between  Measures  for  MOS  64C 


Measure 


Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

.04 

.21 

-.06 

2  Time  in  Service 

.04 

1.00 

.08 

.14 

3  AFQT  Percentile 

.21 

.08 

1.00 

.11 

4  Educational  Attainment 

-.06 

.14 

.11 

1.00 

Intercorrelations 

Between  Measures 

for  MOS  71L 

Measure 

Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

-.02 

.33 

-.06 

2  Time  in  Service 

-.02 

1.00 

-.08 

.05 

3  AFQT  Percentile 

.33 

-.08 

1.00 

-.07 

4  Educational  Attainment 

-.06 

.05 

-.07 

1.00 

Intercorrelations 

Between  Measures 

for  MOS  91A 

Measure 

Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

.05 

.25 

-.05 

2  Time  in  Service 

.05 

1.00 

-.10 

.28 

3  AFQT  Percentile 

.25 

-.10 

1.00 

.02 

4  Educational  Attainment 

-.05 

.28 

.02 

1.00 

Intercorrelations 

Between  Measures 

for  MOS  95B 

Measure 

Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

.06 

.19 

-.10 

2  Time  in  Service 

.06 

1.00 

.00 

.06 

3  AFQT  Percentile 

.19 

.00 

1.00 

-.03 

4  Educational  Attainment 

-.10 

.06 

-.03 

1.00 

Table  5 

Intercorrelations  Corrected  for  Range  Restriction 


Intercorrelations  Between  Measures  for  MOS  IIB 


Measure  1 


1  HOPT  1.00 

2  Time  in  Service  -.07 

3  AFQT  Percentile  .30 

4  Educational  Attainment  -.13 


Measure 

2 

3 

4 

-.07 

.30 

-.13 

1.00 

-.05 

-.05 

-.05 

1.00 

-.08 

-.05 

-.08 

1.00 

Intercorrelations  Between  Measures  for  MOS  13B 


Measure 


Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

.17 

,16 

.00 

2  Time  in  Service 

.17 

1.00 

-.01 

-.04 

3  AFQT  Percentile 

.16 

-.01 

1.00 

.21 

4  Educational  Attainment 

.00 

-.04 

.21 

1.00 

Intercorrelations  Between  Measures  for  MOS  19E 


Measure 


Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

.26 

.29 

.08 

2  Time  in  Service 

.26 

1.00 

-.02 

.09 

3  AFQT  Percentile 

.29 

-.02 

1.00 

.02 

4  Educational  Attainment 

.08 

.09 

.02 

1.00 

Intercorrelations 

Between  Measures  for  MOS  31C 

Measure 

Measure 

1 

2 

3 

4 

1 

HOPT 

1.00 

.02 

.45 

-.03 

2 

Time  in  Service 

.02 

1.00 

.05 

.03 

3 

AFQT  Percentile 

.45 

.05 

1.00 

.05 

4 

Educational  Attainment 

-.03 

.03 

.05 

1.00 

Intercorrelations  Between  Measures  for  MOS  63B 


Measure 


Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

.06 

.11 

.06 

2  Time  in  Service 

.06 

1.00 

-.15 

-.14 

3  AFQT  Percentile 

.11 

-.15 

1.00 

.13 

4  Educational  Attainment 

.06 

-.14 

,13 

1.00 

Table  5  (Continued) 

Intercorrelations  Corrected  for  Range  Restriction 


Intercorrelations  Between  Measures  for  MOS  64C 


Measure 


Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

.06 

.33 

-.02 

2  Time  in  Service 

.06 

1.00 

.13 

.16 

3  AFQT  Percentile 

.33 

.13 

1.00 

.17 

4  Educational  Attainment 

-.02 

.16 

.17 

1.00 

Intercorrelations 

Between  Measures  for  MOS  71L 

Measure 

Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

-.05 

.46 

-.09 

2  Time  in  Service 

-.05 

1.00 

-.12 

.05 

3  AFQT  Percentile 

.46 

-.12 

1.00 

-.11 

4  Educational  Attainment 

-.09 

.05 

-.11 

1.00 

Intercorrelations 

Between  Measures  for  MOS  91A 

Measure 

Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

.01 

.38 

-.04 

2  Time  in  Service 

.01 

1.00 

-.16 

.27 

3  AFQT  Percentile 

.38 

-.16 

1.00 

.03 

4  Educational  Attainment 

-.04 

.27 

.03 

1.00 

Intercorrelations 

Between  Measures 

for  MOS  95B 

Measure 

Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

.06 

.30 

-.11 

2  Time  in  Service 

.06 

1.00 

.00 

.06 

3  AFQT  Percentile 

.30 

.00 

1.00 

-.06 

4  Educational  Attainment 

-.11 

.06 

-.06 

1.00 

AFQT  and  Army  job  performance  is  based  on  our  evaluation  of  five 
job  performance  factors:  General  Soldiering  Proficiency,  Core 
Technical  Proficiency,  Effort  and  Leadership,  Personal  Dis¬ 
cipline,  and  Physical  Fitness /Military  Bearing.  These  factors 
were  derived  empirically  from  a  complete  set  of  soldier  perform¬ 
ance  measures  based  on  hands-on  performance,  written  school  and 
job  knowledge  and  peer/ supervisor  performance  ratings.  The  Army 
project  scientists  along  with  their  scientific  and  policy 
advisors  intended  the  hands-on  test  to  be  only  one  part  of  this 
integrated  three-part  strategy  to  measure  representative  job 
performance.  The  Army’s  project  was  designed  such  that  a 
comprehensive  task  sampling  procedure  was  used  to  define  the 
population  of  tasks  in  each  MOS,  and  then  to  select  30  job  tasks 
to  represent  the  population  of  the  MOS  tasks.  The  task  lists 
were  then  reviewed  by  the  proponent  schools  for  completeness  and 
representativeness  of  the  occupation.  Fifteen  tasks  requiring  a 
high  level  of  physical  skill,  a  series  of  prescribed  steps,  and 
speed  of  performance  were  selected  for  hands-on  testing.  In 
other  words,  the  appropriateness  and  feasibility  of  hands-on 
testing  was  a  consideration  in  the  selection  of  tasks  for  hands- 
on  testing.  Information  about  job  performance  was  also  obtained 
from  written  school  and  job  knowledge  tests  as  well  as  from  a 
variety  of  peer  and  supervisory  ratings.  More  detailed 
information  is  included  in  reports  listed  in  the  reference 
section  of  this  chapter. 

Sample  Description.  The  Army’s  Project  A  sample  has  been 
described  in  detail  in  the  Project’s  Annual  Reports  (see  refer¬ 
ence  list)  as  well  as  earlier  in  this  chapter. 

Reliability  Results.  As  reported  in  the  Fifth  Annual  Report 
to  Congress,  the  split-half  reliability  estimates  for  the  hands- 
on,  job  knowledge,  and  school  knowledge  measures  indicated  a  high 
degree  of  Internal  consistency  in  the  measures  for  each  MOS. 

Similarly,  the  inter-rater  reliability  estimates  for  the 
Behaviorally  Anchored  Rating  Scales  (BARS)  demonstrated  moderate 
to  strong  reliabilities  among  raters.  The  fact  that  the  relia¬ 
bility  estimates  for  supervisor  ratings  are  generally  higher  than 
the  estimates  for  peer  ratings  may  be  attributed  to  the  fact  that 
supervisors  have  more  experience  rating  individuals  than  non¬ 
supervisors. 

Job  Performance  Measures  and  ASVAB  Relationships.  As 
presented  in  the  last  report  to  Congress,  the  intercorrelations 
among  the  criterion  measures  for  each  of  the  nine  MOS  are  all 
non-zero  and,  for  the  most  part,  in  the  moderate  range.  This 
pattern  of  results  is  similar  to  that  found  in  the  field  test 
data.  These  moderate  correlations  Indicate  that  different 
aspects  of  the  job  performance  domain  are  being  measured  by  the 
different  testing  Instruments. 

Although  the  Army  presented  the  results  of  its  concurrent 
validation  in  the  Fifth  Annual  Report  to  Congress,  a  summary  is 


supplied  below.  Briefly,  statistical  techniques  were  employed  to 
determine  criterion  construct  scores.  Analyses  resulted  In  the 
Identification  of  five  criterion  constructs  (factors): 

1.  Basic  Soldiering  Skills  (use  of  basic  weapons,  first 
aid,  etc.) 

2.  MOS  Specific  Technical  Skills  (document  preparation 
for  71L;  tank  operation  for  19E,  etc.) 

3.  Exercise  of  Leadership.  Effort  and  Self  Development 
(the  Individual’s  willingness  to  perform  the  tasks  and  to 
be  cooperative  and  supportive  to  other  soldiers) 

4.  Maintaining  Personal  Discipline  (adherence  to  Army 
regulations  and  traditions,  commitment  to  high  standards 
of  personal  conduct) 

5.  Military  Bearing /Fitness  (maintenance  of  appropriate 
military  appearance  and  good  physical  condition) 

Once  the  scores  comprising  each  criterion  factor  (construct) 
were  Identified,  the  scores  were  weighted  and  summed  within  each 
factor  to  obtain  a  construct  score.  The  correlations  between  the 
construct  scores  and  AFQT  scores  for  the  nine  MOS  are  presented 
In  Table  6  and  7.  The  corrected  correlations  In  Table  7  present 
"a  more  accurate  picture  of  the  validity  of  AFQT,"  as  explained 
In  Chapter  2  under  "Validity  Results. "  It  should  be  noted  that 
for  IIB  all  skills  are  general  soldiering  skills.  Examination  of 
the  correlations  Indicates  that  AFQT  predicts  General  Soldiering 
Proficiency  and  Core  Technical  Proficiency 

Table  6 

Uncorrected  Correlation  Between  AFQT  and  Criterion  Construct  Scores 


Physical 

General  Core  Fitness 

Soldiering  Technical  Effort  and  Personal  Military 

MOS  N  Proficiency  Proficiency  Leadership  Discipline  Bearing 


IIB 

478 

.42 

— 

.19 

.15 

-.03 

13B 

426 

.34 

.23 

.11 

.05 

-.10 

19E 

367 

.46 

.26 

.17 

.13 

-.06 

31C 

274 

.39 

.41 

.08 

.04 

-.16 

63B 

461 

.31 

.24 

.04 

-.01 

-.04 

64C 

486 

.37 

.23 

.00 

-.04 

-.04 

71L 

407 

.40 

.43 

.16 

.10 

.00 

91A 

378 

.31 

.34 

.09 

.07 

-.01 

95B 

569 

.30 

.28 

.13 

.10 

-.01 

Table  7 


Corrected  Correlation  Between  AFQT  and  Criterion  Construct  Scores 


MOS 

N 

General 

Soldiering 

Proficiency 

Core 

Technical 

Proficiency 

Effort  and 
Leadership 

Personal 

Discipline 

Physical 

Fitness 

Military 

Bearing 

IIB 

478 

.58 

.40 

.25 

.07 

13B 

426 

.43 

.33 

.29 

.13 

.00 

19E 

367 

.49 

.23 

.25 

.08 

-.02 

31C 

274 

.59 

.65 

.10 

.00 

00 

CM 

• 

63B 

461 

.48 

.51 

.25 

.02 

-.10 

64C 

486 

.60 

.43 

.10 

-.14 

-.10 

71L 

407 

.57 

.56 

.28 

.01 

.05 

91A 

378 

.68 

.67 

.21 

.05 

-.24 

95B 

569 

.64 

.61 

.40 

.31 

-.12 

Note:  Numbers  are  corrected  correlation  coefficients.  The 

correction  for  range  restriction  employed  the  multi¬ 
variate  correction  procedure  designated  by  the  Joint 
Services  Job  Performance  Measurement  Working  Group. 
(Working  Group  Minutes  of  the  9-10  July  1985  meeting, 
26  August  1985.) 


construct  scores.  The  AFQT  does  not  predict  the  other  three 
construct  scores;  correlations  between  AFQT  scores  and 
Effort /Leadership,  Personal  Discipline  and  Fitness /Bearing  scores 
are  generally  not  statistically  different  from  zero.  These 
results  are  not  unexpected  since  the  AFQT  measure  was  developed 
to  predict  success  in  training  for  first  term  military  enlistment 
(selection)  purposes. 

As  noted  earlier,  the  Army’s  goals  include  the  development 
and  validation  of  new  and/or  improved  selection  and  classifica¬ 
tion  measures.  Preliminary  analyses  of  the  Army  project’s 
additional  (e.g.,  non-cognitive)  predictor  measures  indicate  that 
the  criterion  construct  scores  poorly  predicted  (Effort/Leader¬ 
ship,  Personal  Discipline  and  Physical  Fitness /Military  Bearing) 
by  AFQT  are  well-predicted  by  other  Project  A  measures. 


SUMMARY  AND  CONCLUSIONS 


The  analyses  presented  in  this  chapter  along  with  the  data 
presented  in  the  Fifth  Annual  report  to  Congress  clearly  support 
the  position  that  ASVAB  predicts  the  technical  aspects  of  job 


performance  of  first  term  soldiers,  and,  as  such,  can  be  used  to 
link  enlistment  standards  to  job  performance.  In  fact,  the  Army 
has  already  embarked  upon  research  to  examine  various  linkage 
Issues  and  procedures.  Nevertheless,  perhaps  one  of  the  most 
striking  conclusions  that  can  be  drawn  from  the  Army’s  research 
on  performance  measurement  to  date  is  that  jobs  are  indeed  multi¬ 
dimensional.  While  ASVAB  can  predict  the  general  soldiering  and 
core  technical  (can-do)  dimensions  of  performance,  it  is  not  a 
very  good  predictor  of  these  will-do  dimensions.  It  is  clearly 
important  to  focus  on  all  aspects  of  job  performance  to  optimize 
the  opportunity  to  improve  significantly  enlistment  (selection 
and  classification)  systems. 
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Army 


Service  Research  Efforts  and  Linkage  Issues 
Army  Summary  for  DoD  Chapter 


The  Army  is  engaged  in  two  research  projects  directed  at 
linking  enlistment  standards  to  job  performance.  The  first 
project,  the  Enlisted  Personnel  Allocation  System  (EPAS),  begun 
in  1981  and  scheduled  for  completion  in  1989,  takes  what  could  be 
called  a  macro  approach  to  the  problem,  as  described  below.  The 
second  project,  also  described  below  under  Synthetic  Validation 
began  in  March  1987  and  is  a  three-year  effort  designed  to 
examine  the  feasibility  of  applying  synthetic  validation  and 
standard  setting  procedures  within  the  Army’s  selection  and 
classification  system. 

EPAS.  The  linking  of  soldier  performance  data  to  enlistment 
and  assignment  policy  requires  several  kinds  of  Information: 

1.  The  applicant  population 

2.  The  jobs  or  tasks  that  need  to  be  performed 

3.  Tradeoffs  in  the  payoff  from  assigning  applicants  to 
different  jobs 

First  of  all,  the  information  used  in  the  linking  of 
standards  or  quality  requirements  must  be  compatible  across  all 
alternatives.  That  is,  the  kinds  of  predictors  should  be 
generallzable  across  all  Army  applicants  and  jobs.  There  needs 
to  be  an  assessment  of  the  value  of  increasing  performance  in 
different  jobs  that  can  be  used  to  make  comparisons . 

Second,  some  assessment  is  needed  of  the  population  available 
for  selection  and  assignment.  Policies  need  to  be  evaluated  for 
an  entire  population  against  all  job  requirements,  rather  than 
against  a  small  segment  of  the  population  or  a  limited  number  of 
jobs. 

The  Enlisted  Personnel  Allocation  System  (EPAS)  provides  a 
way  of  analyzing  the  impact  of  alternative  policies  and  recruit 
populations  on  the  Army.  For  example,  different  enlistment 
standards,  high  quality  recruit  supplies,  and  assignment  payoffs 
can  be  simulated  to  find  out  what  would  happen  to  the  Army’s 
enlisted  force  with,  for  example: 

1.  different  quality  distributions 

2.  alternative  quality  pools 

3.  different  enlistment  or  job  standards 


4.  different  predictor  information 

5.  different  job  utility  information 

Synthetic  Validation.  In  addition  to  its  much-desribed  work 
on  Project  A,  and  EPAS  the  Army  is  conducting  a  separate  project 
to  evaluate  procedures  for  linking  job  performance  to  selection 
standards.  The  Army  Synthetic  Validation  Project  (SVP)  is 
motivated  by  the  need  to  establish  selection  standards  for  more 
than  250  entry-level  enlisted  MOS.  Project  A  has  developed 
measures  of  job  performance  for  only  20  of  these  MOS. 

The  objective  of  the  SVP  is  to  evaluate  the  extent  to  which 
job  performance  prediction  equations  can  be  established  syntheti¬ 
cally,  particularly  in  cases  where  empirical  validation  is  not  at 
all  feasible.  The  synthetic  validation  approach  involves 
examining  ways  in  which  performance  in  enlisted  jobs  can  be 
decomposed  into  a  set  of  common  job  components.  In  order  to 
develop  prediction  equations  synthetically,  it  must  be  possible 
to  establish  equations  for  predicting  preformance  on  each 
component  in  the  overall  list  of  job  components,  and  the  equation 
for  each  component  must  generalize  across  all  jobs  for  which  the 
component  is  relevant.  If  these  conditions  hold,  then  overall 
prediction  equations  can  be  established  as  the  weighted  sum  of 
prediction  equations  for  individual  job  components. 

A  second  part  of  the  SVP  concerns  the  development  of  proce¬ 
dures  for  setting  job  performance  standards.  If  such  standards 
can  be  reliably  established,  then  selection  standards  can  be 
based  on  the  expected  probability  of  achieving  acceptable  levels 
of  job  performance. 
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CHAPTER  3 


ARMY  RESEARCH  TO  LINK  STANDARDS  FOR  ENLISTMENT  TO 
ON-THE-JOB  PERFORMANCE 

CURRENT  STATUS  AND  ACCOMPLISHMENTS 


The  Army  Research  Institute  is  currently  engaged  in  a  large- 
scale,  multi-year  project  to  improve  the  Army  selection  and 

classification  system  and,  thereby,  increase  the  overall  effec¬ 
tiveness  of  the  force.  The  goal  of  the  Army’s  program  for 

increasing  the  efficiency  of  enlisted  personnel  selection  and 

utilization  is  to  enable  the  Army  to  meet  its  peacetime  and 

mobilization  missions  through  improved  matching  of  individuals  to 
Military  Occupational  Specialties  (MOS).  The  research  is  aimed 
at  developing  comprehensive  selection  and  classification 
procedures  to  validly  predict  performance  in  Army  training  and 
occupational  specialties. 

A  complete  description  of  the  Army’s  research  program  and 
accomplishments  has  been  published  separately  by  the  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences  in  the 
annual  reports  for  Project  A,  "Improving  the  Selection,  Classifi¬ 
cation  and  Utilization  of  Army  Enlisted  Personnel"  (1983,  1984, 
1985,  1986).  A  detailed  description  of  the  project  also  appears 
in  previous  Annual  Reports  to  Congress  on  Joint-Service  Efforts 
to  Link  Standards  of  Enlistment  to  On- the- Job  Performance  (1983, 
1984,  1985,  1986,  1987). 

The  Army’s  research  focuses  on  19  MOS.  The  MOS  were  selected 
to  be  representative  of  the  Army  and  include  all  operational 
ASVAB  aptitude  area  composites.  Blacks,  Whites,  Hispanics, 
males,  and  females  are  present  in  these  MOS  in  the  same  propor¬ 
tions  as  in  total  accessions.  These  MOS  represent  441  of  annual 
Army  enlistments. 

A  number  of  measures,  including  measures  of  training  success, 
service-wide  job  performance,  and  MOS-specific  hands-on  job 
performance,  were  developed  for  these  MOS.  For  reasons  of  cost 
efficiency,  not  all  measures  were  developed  for  all  19  MOS.  All 
project  criterion  measures  were  developed  for  the  following  MOS 
and  serve  as  the  focus  of  this  report: 

1.  IIB  -  Infantryman 

2.  13B  -  Cannon  Crewman 

3.  19E  -  Tank  Crewman 

4 .  31C  -  Radio  Teletype  Operator 

5.  63B  -  Light  Wheel  Vehicle/Power  Generation  Mechanic 

6.  64C  -  Motor  Transport  Operator 

7.  71L  -  Administrative  Specialist 

8.  91A  -  Medical  Specialist 

9.  95B  -  Military  Police 


^Measures  of  training  success  and  service-wide  performance  were 
developed  for  the  following  specialties: 

1.  12B  -  Combat  Engineer 

2.  16S  -  MANPADS  Crewman 

3.  27E  -  TOW/Dragon  Repairman 

4.  51B  -  Carpentry /Masonry  Specialist 

5.  54E  -  Chemical  Operations  Specialist 

6.  55B  -  Ammunition  Specialist 

7.  67N  -  Utility  Helicopter  Repairman 

8.  76¥  -  Petroleum  Supply  Specialist 

9.  76Y  -  Supply  Specialist 

10.  94B  -  Food  Service  Specialist 

A  complete  report  of  accomplishments  and  current  status  of 
the  Army’s  research  to  link  enlistment  standards  to  job  perform¬ 
ance  has  been  published  by  the  Army  Research  Institute  (1983, 
1984,  1985,  1986)  and  is  available  for  distribution.  What 

follows  includes  a  svimmary  of  the  Army’s  accomplishments  in  the 
area  of  job  performance  measurement. 

As  reported  last  year  in  the  Annual  Report  to  Congress  the 
Army  has  completed  the  concurrent  validation  phase,  i.e.,  joint 
administration  of  predictor  and  criterion  measures,  of  its 
project.  The  research  not  only  established  the  feasibility  of 
measuring  job  performance  but  also  identified  five  performance 
constructs,  described  later  in  this  chapter,  thereby  providing 
support  for  the  notion  that  job  performance  is  multi-dimensional. 
In  addition,  the  research  established  that  there  is  a  positive 
relationship  between  ASVAB  and  job  performance  during  the  first 
term  of  service.  More  specifically,  the  higher  an  applicant’s 
ASVAB  score  the  better  he  or  she  performs  the  technical  or  "can- 
do*  aspects  of  the  job.  The  correlations  ranged  from  .26  to  .70. 
The  "will-do"  or  motivational  factors  are  not  well-predicted  by 
ASVAB,  but  the  Army’s  new  predictor  measures  are  expected  to 
improve  the  prediction  of  the  non-cognitive  job  performance 
dimensions. 

The  Army  is  currently  engaged  in  the  longitudinal  validation 
phase  of  its  project.  Fifty- three  thousand  Army  entrants  were 

administered  the  Army  project’s  predictor  battery  at  eight 
reception  battalions.  Collection  of  training  (school)  knowledge 
measures,  administered  to  soldiers  in  the  sample  at  the  end  of 
Advanced  Individual  Training  (AIT),  ended  in  May  of  1988. 
Administration  of  the  job  performance  measures  is  now  being 
conducted. 

Implementation  of  the  project’s  products  currently  includes 
three  efforts.  First,  with  respect  to  the  use  of  ASVAB, 
modifications  have  been  made  to  the  aptitute  area  composites  to 
improve  can-do  assignment  efficiency.  Forty-four  MOS  are  in  the 
process  of  being  reassigned  to  different  entry  composites  in 
order  to:  increase  validity,  increase  consistency  of  composite 
use  within  a  career  management  field,  and  increase  the  accuracy 


•of  prediction  for  women  and  blacks.  The  second  major  implement¬ 
ation  effort  involves  one  of  the  new  predictors.  Assessment  of 
Background  and  Life  Experiences  (ABLE) ,  a  tempersunent/personal 
history  instrument.  The  Army  is  presently  planning  for  an 
Initial  Operational  Test  and  Evaluation  (lOT&E)  of  ABLE.  ABLE 
will  help  the  Army  to  predict  better  the  job  dimensions  of: 
Effort  and  Leadership,  Personal  Discipline,  and  Physical  Fitness 
and  Military  Bearing.  The  third  implementation  is  designed  to 
better  capitalize  on  individual  psychomotor  and  spatial 
attributes  in  MOS  which  require  them  for  effective  gunnery 
performance. 


RESEARCH  DESIGN 


Types  of  Measures  Developed 

Hands-on  performance  measures,  job  knowledge  tests,  and 
performance  rating  scales  were  developed  for  training  success, 
service-wide  performance,  and  MOS-speclfic  performance  for  the 
MOS  listed  previously.  The  Army’s  rationale  for  the  development 
of  multiple  measures  of  job  performance  is  based  upon  the 
knowledge  that  a  soldier’s  job  is  multi-faceted  and  there  are 
different  aspects  of  job  performance.  Therefore,  the  Army’s 
research  project  has  developed  different  kinds  of  tests  to  assess 
these  different  aspects  of  job  performance  and  thereby  obtain 
information  about  the  domain  of  job  performance  behaviors.  A 
more  complete  description  of  these  measures  appears  in  the 
Project  A  Annual  Reports  (1983,  1984,  1985,  1986)  and  in  previous 
Annual  Reports  to  Congress  (1983,  1984,  1985,  1986). 

Data  Collection  Procedures 


Data  collection  began  with  the  briefing  of  local  military 
commanders,  examination  of  the  test  sites,  equipment  and  sup¬ 
plies,  training  of  test  administrators  and  scorers,  and  a  dry  run 
of  testing  procedures.  Test  site  managers  were  appointed  to 
supervise  the  actual  data  collections  and  were  responsible  for 
controlling  the  quality  and  flow  of  data  from  the  testing. 

The  concurrent  validation  data  collection  began  in  June  and 
ended  in  November  1985.  As  reported  in  the  Fourth  Annual  Report 
to  Congress,  data  were  collected  at  fourteen  different  sites. 
The  data  from  5200  soldiers  in  9  MOS,  who  were  tested  on  all  the 
project  criterion  measures,  were  entered  into  the  research  data 
base.  Data  from  an  additional  4000  soldiers  in  the  10  other  MOS, 
who  were  tested  only  with  the  measures  of  training  success  and 
service-wide  performance  were  also  entered.  Analyses  began  as 
soon  as  all  the  data  were  collected  and  entered  in  the  lon¬ 
gitudinal  research  data  base. 


.RESULTS  OF  DATA  ANALYSES 


Conmon  Data  Analyses 

Sample  Description.  The  analyses  are  based  on  data  collected 
during  the  Army’s  concurrent  validation  data  collection  in  1985. 
Soldiers  were  in  Skill  Level  One,  entry-level  positions  in  their 
respective  MOS  primarily  with  18-24  months  experience  in  the 
Army. 


Table  1  presents  descriptive  statistics  for  the  variables 
included  in  these  analyses.  The  hands-on  performance  test  (HOPT) 
scores  have  been  converted  to  standard  T-scores  in  order  to  allow 
comparison  across  MOS.  Time  in  service  is  the  number  of  months, 
averaged  across  soldiers,  of  Army  service.  In  accordance  with 
the  Army’s  sampling  plan,  the  means  are  between  18  and  24  months, 
with  small  variances  (standard  deviations).  Average  AFQT 
percentile  score  for  each  MOS  sample  is  also  presented,  along 
with  the  number  of  soldiers  classified  as  high  school  diploma 
graduates  (HSDG)  or  non-high  school  diploma  graduates  (NHSDG). 

Table  2  also  presents  descriptive  statistics  for  the  sample, 
this  time  broken  into  four  groups  for  time  in  service  and  two 
AFQT  groups,  upper  50  percentiles  (I-IIIA)  and  lower  50 
percentiles  (IIIB-IV).  Again,  given  the  design  of  the  Army’s 
project,  few  soldiers  were  expected  in  the  1-12,  25-36,  and  37+ 
Time-in-Service  groups.  Therefore,  cells  with  fewer  than  25 
observations  have  been  marked  "N/A" . 

Reliability  Results.  Two  indices  of  reliability,  split-half 
and  coefficient  alpha,  for  HOPT  were  computed  and  are  presented 
in  Table  3.  Both  uncorrected  and  corrected  reliability  estimates 
are  included  along  with  the  standard  error  of  measurement.  The 
table  also  includes  the  number  of  tasks  in  the  HOPT  scores  for 
these  common  data  analyses. 

Hands-On  Performance  Test.  Aptitude.  Education,  and  Job 
Experience  Relationships .  Intercorrelations  among  the  measures 
are  presented  for  each  MOS.  Uncorrected  correlations  are 
depicted  in  Table  4.  The  correlations  in  Table  5  and  elsewhere, 
as  noted  in  this  chapter,  have  been  corrected  for  range  restric¬ 
tion  using  a  multivariate  correction  procedure  as  agreed  upon  by 
the  Joint  Service  Job  Performance  Measurement  Working  Group  for 
the  purposes  of  these  analyses. 

Service-specific  Data  Analyses 

Simply  for  purposes  of  comparison.  Table  6  presents  the 
correlations  between  the  hands-on  performance  test  scores  and  the 
Army’s  ASVAB  classification  (aptitude  area  or  AA)  composite  for 
each  of  the  nine  MOS.  Since  the  AA  composites  are  used  to 
classify  individuals  for  entry  into  specific  occupations,  it  is 


Table  1 


Descriptive  Statistics  for  HOPT,  Time  in  Service, 
Aptitude,  and  Educational  Attainment 


HOPT  Time  Educational 


MOS 

(Standard 

T-Score) 

in  Service 
(Months ) 

AFQT 

(Percentile) 

Attainment 

(N) 

Mean 

50 

20.4 

55.9 

423 

HSDG 

IIB 

SD 

10 

5.6 

21.8 

55 

NHSD6 

N 

491 

478 

491 

Mean 

50 

21.8 

46.8 

353 

HSDG 

13B 

SD 

10 

6.1 

20.2 

73 

NHSDG 

N 

434 

425 

464 

Mean 

50 

19.4 

54.7 

317 

HSDG 

19E 

SD 

10 

4.6 

22.0 

50 

NHSDG 

N 

378 

367 

394 

Mean 

50 

20.5 

56.9 

245 

HSDG 

31C 

SD 

10 

5.6 

20.9 

29 

NHSDG 

N 

275 

274 

289 

Mean 

50 

19.9 

49.6 

397 

HSDG 

63B 

SD 

10 

5.0 

19.6 

64 

NHSDG 

N 

443 

461 

478 

Mean 

50 

20.0 

42.8 

420 

HSDG 

64C 

SD 

10 

4.6 

19.5 

66 

NHSDG 

N 

481 

486 

507 

Mean 

50 

22.3 

54.1 

401 

HSDG 

71L 

SD 

10 

4.5 

20.3 

7 

NHSDG 

N 

416 

408 

427 

Mean 

50 

20.6 

61.2 

310 

HSDG 

91A 

SD 

10 

5.4 

16.9 

68 

NHSDG 

N 

392 

378 

392 

Mean 

50 

21.3 

63.4 

525 

HSDG 

95B 

SD 

10 

5.0 

15.1 

44 

NHSDG 

N 

577 

569 

597 

Table  2 


Mean  HOPT  Scores  (Standardized  T-Scores)  by  Aptitude 
and  Job  Experience  Levels  for  9  Jobs 


MOS 

IIB 

MOS 

13B 

MOS 

19E 

Time  in 
Service 
(Months ) 

AFQT 

I-IIIA 

AFQT 

IIIB-IV 

AFQT 

I-IIIA 

AFQT 

IIIB-IV 

AFQT 

I-IIIA 

AFQT 

IIIB-IV 

1-12 

Mean 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

SD 

M 

_  «  _ 

... 

... 

... 

... 

N 

7 

6 

1 

5 

7 

4 

Mean 

51.8 

48.1 

49.9 

47.9 

51.0 

46.3 

13-24 

SD 

10.3 

9.6 

10.1 

9.5 

9.0 

11.1 

N 

214 

154 

115 

151 

163 

121 

Mean 

50.8 

48.3 

53.4 

52.3 

55.7 

51.1 

25-36 

SD 

9.4 

9.3 

11.4 

8.9 

7.1 

9.0 

N 

42 

46 

46 

66 

32 

25 

37+ 

Mean 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

SD 

—  —  — 

..... 

... 

—  —  — 

... 

... 

N 

6 

3 

5 

6 

0 

1 

Mean 

51.5 

47.9 

50.8 

49.4 

52.2 

46.9 

Total 

SD 

10.0 

9.7 

10.5 

9.6 

9.0 

10.9 

Group 

N 

269 

209 

167 

228 

202 

151 

MOS 

31C 

MOS 

636 

MOS 

64C 

Time  in 

Service 

AFQT 

AFQT 

AFQT 

AFQT 

AFQT 

AFQT 

(Months ) 

I-IIIA 

IIIB-IV 

I-IIIA 

IIIB-IV 

I-IIIA 

IIIB-IV 

1-12 

Mean 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

SD 

.  ^ 

... 

... 

... 

... 

... 

N 

10 

5 

6 

7 

4 

7 

Mean 

51.9 

47.6 

51.3 

48.3 

51.7 

48.4 

13-24 

SD 

10.0 

9.1 

9.9 

9.9 

9.6 

9.9 

N 

136 

78 

172 

177 

106 

248 

Mean 

N/A 

N/A 

N/A 

51.9 

55.1 

49.7 

25-36 

SD 

— 

— 

10.3 

8.6 

10.8 

N 

17 

9 

16 

46 

36 

60 

37+ 

Mean 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

SD 

... 

... 

... 

... 

N 

2 

3 

1 

3 

0 

0 

Mean 

51.6 

47.0 

51.1 

49.1 

52.7 

48.7 

Total 

SD 

10.3 

8.9 

10.1 

9.9 

9.5 

10.0 

Group 

N 

165 

95 

195 

233 

146 

315 

Table  2  (Continued) 


Mean  HOPT  Scores  (Standardized  T-Scores)  by  Aptitude 
and  Job  Experience  Levels  for  9  Jobs 


MOS 

71L 

MOS  91A 

MOS  95B 

Time  in 
Service 
(Months) 

AFQT 

I-IIIA 

AFQT 

IIIB-IV 

AFQT  AFQT 

I-IIIA  IIIB-IV 

AFQT  AFQT 

I-IIIA  IIIB-IV 

1-12 

Mean 

SD 

N 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

7 

3 

10 

4 

19 

5 

Mean 

52.9 

47.2 

51.1 

47.1 

50.1 

48.1 

13-24 

SD 

9.7 

9.8 

9.7 

10.3 

10.1 

9.7 

N 

173 

125 

209 

74 

317 

84 

Mean 

52.4 

47.1 

50.4 

N/A 

52.0 

49.3 

25-36 

SD 

8.5 

8.7 

10.5 

— 

10.5 

8.7 

N 

45 

42 

46 

22 

99 

25 

Mean 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

37+ 

SD 

— 

— 

— 

— 

— 

N 

2 

1 

2 

1 

1 

0 

Mean 

52.8 

47.1 

51.1 

46.9 

50.5 

48.3 

Total 

SD 

9.5 

9.6 

9.8 

9.9 

10.1 

9.4 

Group 

N 

227 

171 

277 

101 

436 

114 

Table  3 


Reliability  Estimates  and  Standard  Errors  of  Measurement 
for  Hands-on  Performance  Tests 


MOS 

Number  of 
Tasks 

Split- 

Half 

Coefficient 

Alpha 

SEM® 

IIB 

14 

ucoef 

.56 

.55 

6.71 

ccoef 

.57 

.57 

13B 

17 

ucoef . 

.76 

.70 

5.52 

ccoef . 

.76 

.70 

19E 

15 

ucoef. 

.57 

.56 

6.66 

ccoef. 

.59 

.57 

31C 

15 

ucoef. 

.81 

.69 

5.58 

ccoef. 

.83 

.72 

63B 

15 

ucoef. 

.47 

.40 

7.74 

ccoef. 

.47 

.40 

64C 

16 

ucoef. 

.66 

.60 

6.32 

ccoef. 

.68 

.63 

71L 

15 

ucoef. 

.73 

.68 

5.65 

ccoef. 

.76 

.72 

91A 

17 

ucoef. 

.59 

.72 

5.31 

ccoef. 

.62 

.74 

95B 

16 

ucoef. 

.63 

.52 

6.90 

ccoef. 

.65 

.55 

^Standard  Error  of  Measurement  calculated  with  the  Coefficient 
Alpha  reliability  estimates  and  HOPT  standard  T-Score  standard 
deviations . 

^ncorrected  reliability  estimate. 

*^Corrected  reliability  estimate. 


Table  4 

Uncorrected  Intercorrelations 


Intercorrelations  Between  Measures  for  MOS  IIB 


Measure 


Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

-.06 

.25 

.12 

2  Time  in  Service 

-.06 

1.00 

-.01 

.05 

3  AFQT  Percentile 

.25 

-.01 

1.00 

.08 

4  Educational  Attainment® 

.12 

.05 

.08 

1.00 

Intercorrelations 

Between  Measures 

for  MOS  13B 

Measure 

Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

.18 

.13 

.02 

2  Time  in  Service 

.18 

1.00 

-.01 

.04 

3  AFQT  Percentile 

.13 

-.01 

1.00 

-.11 

4  Educational  Attainment 

.02 

.04 

-.11 

1.00 

Intercorrelations 

Between  Measures 

for  MOS  19E 

Measure 

Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

.26 

.26 

-.08 

2  Time  in  Service 

.26 

1.00 

-.02 

-.08 

3  AFQT  Percentile 

.26 

-.02 

1.00 

.01 

4  Educational  Attainment 

-.08 

-.08 

.01 

1.00 

Intercorrelations 

Between  Measures 

for  MOS  31C 

Measure 

Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

.01 

.34 

.04 

2  Time  in  Service 

.01 

1.00 

-.02 

-.03 

3  AFQT  Percentile 

.34 

-.02 

1.00 

-.02 

4  Educational  Attainment 

.04 

-.03 

-.02 

1.00 

Intercorrelations 

Between  Measures 

for  MOS  63B 

Measure 

Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

.07 

.13 

-.05 

2  Time  in  Service 

.07 

1.00 

-.11 

.12 

3  AFQT  Percentile 

.13 

-.11 

1.00 

-.08 

4  Educational  Attaiiunent 

-.05 

.12 

-.08 

1.00 

Table  4  (Continued) 
Uncorrected  Intercorrelations 


Intercorrelations  Between  Measures  for  MOS  64C 


Measure 


Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

.04 

.24 

.06 

2  Time  in  Service 

.04 

1.00 

.07 

-.11 

3  AFQT  Percentile 

.24 

.07 

1.00 

-.10 

4  Educational  Attainment 

.06 

-.11 

-.10 

1.00 

Intercorrelations 

Between  Measures  for  MOS  71L 

Measure 

Measure 

1 

2 

3 

4 

1 

HOPT 

1.00 

-.02 

.35 

.06 

2 

Time  in  Service 

-.02 

1.00 

-.10 

-.05 

3 

AFQT  Percentile 

.35 

-.10 

1.00 

.05 

4 

Educational  Attainment 

.06 

-.05 

.05 

1.00 

Intercorrelations 

Between  Measures  for  MOS  91A 

Measure 

Measure 

1 

2 

3 

4 

1 

HOPT 

1,00 

.04 

.28 

.05 

2 

Time  in  Service 

•  04 

1.00 

-.08 

-.28 

3 

AFQT  Percentile 

.28 

-.08 

1.00 

.04 

4 

Educational  Attainment 

.05 

-.28 

.04 

1.00 

Intercorrelations 

Between  Measures  for  MOS  95B 

Measure 

Measure 

1 

2 

3 

4 

1 

HOPT 

1.00 

.06 

.23 

.11 

2 

Time  in  Service 

.06 

1.00 

-.01 

-.06 

3 

AFQT  Percentile 

.23 

-.01 

1.00 

.09 

4 

Educational  Attainment 

.11 

-.06 

.09 

1.00 

Note .  ®For  purposes  of  these  analyses  the  coding  for  Educational 
Attainment  was:  1  =  Non-high  school  degree;  2  =  High  school 
degree. 


Table  5 

Intercorrelations  Corrected  for  Range  Restriction 


Intercorrelations  Between  Measures  for  MOS  IIB 


Measure 


Measure 

1  HOPT 

2  Time  in  Service 

3  AFQT  Percentile 

4  Educational  Attainment® 

Intercorrelations 

12  3 

1.00  -.08  .34 

-.08  1.00  -.04 

.34  -.04  1.00 

-.14  .04  .13 

Between  Measures  for  MOS  13B 

Measure 

4 

.14 

.04 

.13 

1.00 

Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

.18 

.15 

.02 

2  Time  in  Service 

.18 

1.00 

.01 

.04 

3  AFQT  Percentile 

.15 

.01 

1.00 

-.09 

4  Educational  Attainment 

.02 

.04 

-.09 

1.00 

Intercorrelations 

Between  Measures 

for  MOS  19E 

Measure 

Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

.25 

.31 

-.06 

2  Time  in  Service 

.25 

1.00 

.00 

-.08 

3  AFQT  Percentile 

.31 

.00 

1.00 

.02 

4  Educational  Attainment 

-.06 

-.08 

.02 

1.00 

Intercorrelations 

Between  Measures 

for  MOS  31C 

Measure 

Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

.02 

.51 

.06 

2  Time  in  Service 

.02 

1.00 

.01 

-.03 

3  AFQT  Percentile 

.51 

.01 

1.00 

.04 

4  Educational  Attainment 

.06 

-.03 

.04 

1.00 

Intercorrelations 

Between  Measures 

for  MOS  63B 

Measure 

Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

.05 

.13 

-.07 

2  Time  in  Service 

.05 

1.00 

-.14 

.13 

3  AFQT  Percentile 

.13 

-.14 

1.00 

.13 

4  Educational  Attainment 

-.07 

.13 

-.08 

1.00 

Table  5  (Continued) 

Intercorrelations  Corrected  for  Range  Restriction 


Intercorrelations  Between  Measures  for  MOS  64C 


Measure 


Measure 

1 

2 

3 

4 

1  HOPT 

1.00 

.09 

.39 

.01 

2  Time  in  Service 

.09 

1.00 

.12 

-.13 

3  AFQT  Percentile 

.39 

.12 

1.00 

-.11 

4  Educational  Attainment 

.01 

-.13 

-.11 

1.00 

Intercorrelations 

Between  Measures 

for  MOS 

71L 

Measure 

Measure 

1 

2 

3 

4 

1 

HOPT 

1.00 

-.09 

.49 

.11 

2 

Time  in  Service 

-.09 

1.00 

-.19 

.07 

3 

AFQT  Percentile 

.49 

-.19 

1.00 

.12 

4 

Educational  Attainment 

.11 

-.07 

.12 

1 

.00 

Intercorrelations 

Between  Measures 

for  MOS 

91A 

Measure 

Measure 

1 

2 

3 

4 

1 

HOPT 

1.00 

-.01 

.46 

.03 

2 

Time  in  Service 

-.01 

1.00 

-.15 

.28 

3 

AFQT  Percentile 

.46 

-.15 

1.00 

.00 

4 

Educational  Attainment 

.03 

-.28 

.00 

1 

.00 

Intercorrelations 

Between  Measures 

for  MOS 

95B 

Measure 

Measure 

1 

2 

3 

4 

1 

HOPT 

1.00 

.07 

.49 

.13 

2 

Time  in  Service 

.07 

1.00 

.01 

.06 

3 

AFQT  Percentile 

.49 

.01 

1.00 

.12 

4 

Educational  Attainment 

.13 

-.06 

.12 

1 

.00 

Note .  ®For  purposes  of  these  analyses  the  coding  for  Educational 
Attainment  was:  1  =  Non-high  school  degree;  2  =  High  school 
degree. 


not  surprising  that  these  correlations  are  somewhat  higher  than 
the  correlations  between  the  hands-on  scores  and  AFQT. 

Table  6 

Correlations  of  AA  Classification  Composites  and  HOPT 


MOS 

AA  Composite 

N 

Uncorrected 

Correlation 

Corrected 

Correlation 

IIB 

CO 

478 

.31 

.41 

13B 

FA 

395 

.18 

.20 

19E 

CO 

353 

.29 

.37 

31C 

sc 

260 

.36 

.53 

63B 

MM 

428 

.24 

.28 

64C 

OF 

461 

.37 

.52 

71L 

CL 

398 

.37 

.50 

91A 

ST 

378 

.35 

.57 

95B 

ST 

550 

.32 

.57 

We  believe,  however,  that  the  most  appropriate  data  showing 
the  link  between  ASVAB  and  Army  job  performance  is  based  on  our 
evaluation  of  five  job  performance  factors:  General  Soldiering 
Proficiency,  Core  Technical  Proficiency,  Effort  and  Leadership, 
Personal  Discipline,  and  Physical  Fitness /Military  Bearing. 
These  factors  were  derived  empirically  from  a  complete  set  of 
soldier  performance  measures  based  on  hands-on  performance, 
written  school  and  job  knowledge  and  peer/ supervisor  performance 
ratings.  The  Army  project  scientists  along  with  their  scientific 
and  policy  advisors  intended  the  hands-on  test  to  be  only  one 
part  of  this  integrated  three-part  strategy  to  measure  represen¬ 
tative  job  performance.  The  Army’s  project  was  designed  such 
that  a  comprehensive  task  sampling  procedure  was  used  to  define 
the  population  of  tasks  in  each  MOS,  and  then  to  select  30  job 
tasks  to  represent  the  population  of  the  MOS  tasks.  The  task 
lists  were  then  reviewed  by  the  proponent  schools  for  complete¬ 
ness  and  representativeness  of  the  occupation.  Fifteen  tasks 
requiring  a  high  level  of  physical  skill,  a  series  of  prescribed 
steps ,  and  speed  of  performance  were  selected  for  hands-on 
testing.  In  other  words,  the  appropriateness  and  feasibility  of 
hands-on  testing  was  a  consideration  in  the  selection  of  tasks 
for  hands-on  testing.  Information  about  job  performance  was  also 
obtained  from  written  school  and  job  knowledge  tests  as  well  as 
from  a  variety  of  peer  and  supervisory  ratings.  More  detailed 
information  is  included  in  reports  listed  in  the  reference 
section  of  this  chapter. 

Sample  Description.  The  Army’s  Project  A  sample  has  been 


described  in  detail  in  the  Project’s  Annual  Reports  (see  refer¬ 
ence  list)  as  well  as  earlier  in  this  chapter. 

Reliability  Results.  As  reported  in  the  Fifth  Annual  Report 
to  Congress,  the  split-half  reliability  estimates  for  the  hands- 
on,  job  knowledge,  and  school  knowledge  measures  indicated  a  high 
degree  of  internal  consistency  in  the  measures  for  each  MOS. 

Similarly,  the  inter-rater  reliability  estimates  for  the 
Behaviorally  Anchored  Rating  Scales  (BARS)  demonstrated  moderate 
to  strong  reliabilities  among  raters.  The  fact  that  the  relia¬ 
bility  estimates  for  supervisor  ratings  are  generally  higher  than 
the  estimates  for  peer  ratings  may  be  attributed  to  the  fact  that 
supervisors  have  more  experience  rating  individuals  than  non¬ 
supervisors. 

Job  Performance  Measures  and  ASVAB  Relationships.  As 
presented  in  the  last  two  reports  to  Congress,  the  intercorrela¬ 
tions  among  the  criterion  measures  for  each  of  the  nine  MOS  are 
all  non-zero  and,  for  the  most  part,  in  the  moderate  range.  This 
pattern  of  results  is  similar  to  that  found  in  the  field  test 
data.  These  moderate  correlations  indicate  that  different 
aspects  of  the  job  performance  domain  are  being  measured  by  the 
different  testing  instruments. 

Although  the  Army  presented  the  results  of  its  concurrent 
validation  in  the  Fifth  Annual  Report  to  Congress,  a  summary  is 
supplied  below.  Briefly,  statistical  techniques  were  employed  to 
determine  criterion  construct  scores.  Analyses  resulted  in  the 
identification  of  five  criterion  constructs  (factors); 

1.  Basic  Soldiering  Skills  (use  of  basic  weapons,  first 
aid,  etc.) 

2.  MOS  Specific  Technical  Skills  (document  preparation 
for  7IL5  tank  operation  for  19E,  etc.) 

3 .  Exercise  of  Leadership.  Effort  and  Self  Development 
(the  individual’s  willingness  to  perform  the  tasks  and  to 
be  cooperative  and  supportive  to  other  soldiers) 

4.  Maintaining  Personal  Discipline  (adherence  to  Army 
regulations  and  traditions,  commitment  to  high  standards 
of  personal  conduct) 

5.  Military  Bearing /Fitness  (maintenance  of  appropriate 
military  appearance  and  good  physical  condition) 

Once  the  scores  comprising  each  criterion  factor  (construct) 
were  identified,  the  scores  were  weighted  and  summed  within  each 
factor  to  obtain  a  construct  score.  The  correlations  between  the 
construct  scores  and  AFQT  scores  for  the  nine  MOS  are  presented 
in  Table  7  and  8.  It  should  be  noted  that  for  IIB  all  skills  are 
general  soldiering  skills.  Examination  of  the  correlations 


-indicates  that  AFQT  predicts  General  Soldiering  Proficiency  and 
Core  Technical  Proficiency  construct  scores.  The  AFQT  does  not 

Table  7 

Uncorrected  Correlation  Between  AFQT  and  Criterion  Construct  Scores 


Physical 

General  Core  Fitness 

Soldiering  Technical  Effort  and  Personal  Military 


MOS  N  Proficiency  Proficiency  Leadership  Discipline  Bearing 


IIB 

478 

.46 

.19 

.16 

-.06 

13B 

395 

.35 

.21 

.09 

.03 

-.11 

19E 

353 

.48 

.40 

.20 

.15 

-.05 

31C 

260 

.45 

.41 

.12 

.05 

-.14 

63B 

428 

.35 

.31 

.04 

.04 

-.05 

64C 

461 

.42 

.28 

.05 

.00 

-.06 

71L 

398 

.42 

.45 

.17 

.13 

.01 

91A 

378 

.36 

.38 

.07 

.09 

-.07 

95B 

550 

.34 

.29 

.07 

.07 

-.07 

Table  8 

Corrected  Correlation  Between  AFQT  and  Criterion  Construct  Scores 


Physical 


MOS 

N 

General 

Soldiering 

Proficiency 

Core 

Technical 

Proficiency 

Effort  and 
Leadership 

Personal 

Discipline 

Fitness 

Military 

Bearing 

IIB 

478 

.57 

.28 

.24 

-.05 

13B 

395 

.43 

.26 

.20 

.12 

.02 

19E 

353 

.56 

.45 

.29 

.20 

-.02 

31C 

260 

.56 

.60 

.16 

.05 

-.21 

63B 

428 

.45 

.39 

.14 

.10 

1 

o 

64C 

461 

.58 

.39 

.21 

.06 

.00 

71L 

398 

.51 

.56 

.21 

.09 

.07 

91A 

378 

.58 

.60 

.16 

.16 

-.10 

95B 

550 

.64 

.57 

.26 

.22 

-.12 

predict  the  other  three  construct  scores;  correlations  between 
AFQT  scores  and  Effort/Leadership,  Personal  Discipline  and 
Fitness /Bearing  scores  are  generally  not  statistically  different 
from  zero.  These  results  are  not  unexpected  since  the  AFQT 
measure  was  developed  to  predict  success  in  training  for  first 
term  military  enlistment  (selection)  purposes. 


similarly,  Tables  9  and  10  present  the  correlations  between 
the  AA  Composite  Scores  and  the  Army’s  Criterion  Construct 
Scores.  Again,  the  ASVAB  AA  scores  predict  the  two  proficiency 
constructs  better  than  the  remaining  three  constructs. 

Table  9 

Uncorrected  Correlation  Between  AA  Classification 
Composite  and  Criterion  Construct  Scores 


MOS  N 

AA 

General 

Soldiering 

Proficiency 

Core 

Technical 

Proficiency 

Effort  & 
Leader¬ 
ship 

Physical 
Fitness/ 
Personal  Military 
Discipline  Bearing 

IIB  A78 

CO 

.48 

.. 

.23 

.20 

-.03 

13B  395 

FA 

.35 

.24 

.16 

.09 

-.08 

19E  353 

CO 

.47 

.42 

.26 

.13 

.01 

31C  260 

SC 

.46 

.40 

.19 

.10 

.00 

63B  428 

MM 

.33 

.51 

.20 

.07 

-.13 

64C  461 

OF 

.49 

.36 

.18 

.02 

-.06 

71L  398 

CL 

.43 

.46 

.18 

.14 

.00 

91A  378 

ST 

.45 

.44 

.10 

.08 

-.13 

95B  550 

ST 

.41 

.29 

.09 

.09 

-.09 

Table  10 

Corrected  Correlation  Between  AA  Classification 

Composite  and  Criterion  Construct  Scores 

Physical 

General 

Core 

Effort  & 

Fitness/ 

Soldiering 

Technical 

Leader- 

Personal 

Military 

MOS 

N 

AA 

Proficiency 

Proficiency 

ship 

Discipline  Bearing 

IIB 

00 

CO 

.60 

.32 

.27 

-.03 

13B 

395 

FA 

.44 

.29 

.25 

.16 

.03 

19E 

353 

CO 

.57 

.51 

.34 

.16 

.02 

31C 

260 

sc 

.58 

.59 

.23 

.10 

-.23 

63B 

428 

MM 

.44 

.62 

.29 

.13 

-.13 

64C 

461 

OF 

.65 

.49 

.33 

.07 

-.07 

71L 

398 

CL 

.52 

.57 

.22 

.10 

.08 

91A 

378 

ST 

.64 

.64 

.19 

.15 

-.15 

95B 

550 

ST 

.70 

.57 

.28 

.23 

-.14 

As  noted  earlier,  the  Army’s  goals  include  the  development 
and  validation  of  new  and/or  improved  selection  and  classifica¬ 
tion  measures.  Analyses  of  the  Army  project’s  additional  (e.g., 
non-cognitive)  predictor  measures  indicate  that  the  criterion 


i 


construct  scores  poorly  predicted  (Effort/Leadership,  Personal 
Discipline  and  Physical  Fitness /Military  Bearing)  by  ASVAB  are 
well-predicted  by  other  Project  A  measures. 


SUMMARY  AND  CONCLUSIONS 


The  analyses  presented  in  this  chapter  along  with  the  data 
presented  in  the  Fifth  and  Sixth  Annual  reports  to  Congress 
clearly  support  the  position  that  ASVAB  predicts  the  technical 
aspects  of  job  performance  of  first  term  soldiers,  and,  as  such, 
can  be  used  to  link  enlistment  standards  to  job  perfonnance.  In 
fact,  the  Army  has  already  embarked  upon  research  to  examine 
various  linkage  issues  and  procedures.  Nevertheless,  perhaps  one 
of  the  most  striking  conclusions  that  can  be  drawn  from  the 
Army’s  research  on  performance  measurement  to  date  is  that  jobs 
are  indeed  multi-dimensional.  Uhile  ASVAB  can  predict  the 
general  soldiering  and  core  technical  (can-do)  dimensions  of 
performance,  it  is  not  a  very  good  predictor  of  the  will-do 
dimensions.  It  is  clearly  important  to  focus  on  all  aspects  of 
job  performance  to  optimize  the  opportunity  to  improve  sig¬ 
nificantly  enlistment  (selection  and  classification)  systems. 

Emerging  Applications 

In  addition  to  the  research  presented  in  this  report,  the 
Army  is  also  studying  the  relationship  between  enlistment 
standards  and  unit  readiness.  Unit  performance  is  being  examined 
in  these  specialties:  Field  Artillery,  Armor,  Infantry,  Air 

Defense  "Maintenance",  and  Signal.  Preliminary  results  Indicate 
positive  relationships  between  unit  performance  and  enlistment 
standards,  particularly  AFQT  scores,  consistent  with  the  findings 
from  Project  A.  These  research  efforts  are  discussed  in  the  Army 
Appendix  to  this  Chapter. 
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INTRODUCTION 


The  Figural  Reasoning  and  Mazes  tests  are  two  of  the 
cognitive  paper-and-pencil  measures  developed  in  the  Army's 
Project  A  to  assess  abilities  of  potential  soldiers  not  covered 
by  the  general  cognitive  domain  of  the  Armed  Services  Vocational 
Aptitude  Battery  (ASVAB) .  Factor  analyses  conducted  during  the 
construction  of  Project  A  predictor  composites  (cf.  Campbell, 

1988)  found  that  both  instruments  loaded  on  a  common  factor, 
termed  "spatial  abilities";  however,  these  tests  appear  to  tap 
somewhat  different  domains  of  abilities.  The  figural  reasoning 
test  appears  to  measure  general  intelligence,  in  terms  of 
inductive  reasoning  and  working  memory.  In  contrast,  the  mazes 
test  has  been  classified  by  some  (but  not  all)  experts  as  a  test 
of  spatial  visualization. 

A  larger  investigation  (Busciglio,  in  preparation)  assessed 
the  utility  of  the  Project  A  tests,  including  Figural  Reasoning 
and  Mazes,  for  incrementing  the  validity  of  ASVAB  across  a 
variety  of  criterion  measures  in  nine  high-fill  Military 
Occupational  Specialities  (MOS) ,  or  Army  jobs.  Project  A 
criteria  used  were  the  total  score  on  written  tests  (summed 
across  the  School  and  Job  Knowledge  tests)  and  hands-on  tests 
(summed  across  all  hands-on  tasks) ,  as  well  as  scores  on  selected 
individual  tasks  in  the  written  (e.g..  Target  Identification)  and 
hands-on  (e.g..  Navigation,  Operating  Howitzer  Sights,  and  Tank 
Gunnery)  modes.  Also  included  were  two  performance  constructs 
from  Project  A  which  divided  the  written  and  hands-on  tasks  into 
two  composite  scores:  General  Soldiering  Proficiency  and  Core 
Technical  (i.e.,  MOS-Specific)  Proficiency. 

In  addition  to  these  outcome  measures  from  Project  A,  scores 
from  written  tests  developed  annually  by  the  Army  were  used. 

These  instruments.  Skill  Qualification  Tests  (SQT) ,  are  MOS- 
specific  measures  administered  to  incumbents  periodically  to 
certify  their  continuing  job  proficiency.  Although  the  SQT 
covers  the  same  skill  domains  as  the  Project  A  criteria,  sampling 
of  the  tasks  in  these  domains  differs  in  unsystematic  ways  from 
that  used  in  Project  A. 


METHOD 

In  the  larger  effort,  a  series  of  stepwise  multiple 
regression  analyses  were  conducted  separately  for  each  criterion 
measure  and  MOS  for  which  hands-on  testing  was  done.  The  SPSS-X 
regression  procedure  that  was  employed  made  it  possible  to 
control  for  restriction-of-range  in  the  ASVAB  subtests  used  by 
the  Army  for  selection  and  classification,  as  well  as  any 
implicit  restriction  in  the  range  of  predictors  or  criteria  which 
are  correlated  with  these  ASVAB  subtests.  This  was  accomplished 
by  using  as  input  a  matrix  of  estimated  population  (i.e., 
unrestricted)  covariances  among  all  predictors  and  criteria. 
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employing  the  Lawley  correction  formula  presented  in  Lord  and 
Novick  (1968,  pp.  144-148). 

The  stepwise  analyses  first  determined  an  optimal  prediction 
equation  for  ASVAB  scores  by  entering  all  nine  subtests  as  a 
block,  then  using  a  backward  elimination  technique  to  remove 
subtests  that  were  not  individually  significant  (at  the  .05 
level) .  Then,  blocks  of  either  spatial  or  perceptual-psychomotor 
tests  were  added  and  the  backward  elimination  procedure 
determined  which  of  these  added  individually  to  the  R  for  the 
ASVAB  scores.  To  ensure  that  these  predictors  showed  true 
incremental  validity,  the  ASVAB  scores  found  to  be  significant  in 
the  initial  equations  were  not  subject  to  removal  when  the 
Project  A  tests  were  added.  In  the  final  stage  of  analysis, 
spatial  tests  were  added  to  the  equations  containing  significant 
ASVAB  and  perceptual -psychomotor  scores,  and  perceptual- 
psychomotor  tests  were  added  to  equations  composed  of  significant 
ASVAB  and  spatial  scores.  Once  again,  the  added  tests  were 
retained  only  if  individually  significant,  while  the  predictors 
already  in  the  equation  were  not  subject  to  backward  elimination. 


RESULTS 

Using  the  above  procedures,  the  incremental  utility  of 
individual  Project  A  tests  was  assessed  separately  for  each  MOS. 
Table  1  shows  results  for  the  Figural  Reasoning  and  Mazes  tests, 
in  terms  of  individual  semi— partial  correlations  for  the  ^ 
significant  predictors.  For  each  criterion  and  MOS,  predictors 
can  be  significant  in  two  equations;  at  the  second  stage  of 
analysis  when  added,  with  the  other  spatial  tests,  to  the 
equations  containing  only  significant  ASVAB  subtests,  and  at  the 
third  stage,  when  added  to  significant  ASVAB  and  perceptual- 
psychomotor  scores. 

As  Table  1  shows,  the  Mazes  test  was  found  to  be  a  poor 
incremental  predictor,  across  all  criteria  and  MOS.  In  contrast, 
the  Figural  Reasoning  test  appeared  to  be  highly  useful  for 
improving  the  prediction  of  quite  a  few  criterion  measures, 
particularly  Total  Score  on  School  and  Job  Knowledge  Tests,  Total 
Score  on  Hands-On  Tests,  General  Soldiering  Proficiency,  Core 
Technical  Proficiency,  and  Determining  Grid  Coordinates. 

The  positive  results  for  the  Figural  Reasoning  test  must  be 
viewed  in  light  of  its  possible  redundancy  with  another  strong 
spatial  predictor.  Assembling  Objects.  For  example,  Peterson 
(1987)  found  that  the  two  tests  showed  the  highest 
intercorrelation  among  the  six  spatial  ability  tests  and  loaded 
highly  on  the  same  factor  when  spatial  tests  were  factor  analyzed 
(see  pp.  81,  88-89  from  the  Army's  report  to  the  Technical 
Advisory  Service  Panel).  The  redundancy  between  the  two  tests 
was  assessed  by  examining  the  Busciglio  (in  preparation)  results 
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to  determine  the  number  of  MOS  for  which  both  tests  were 
significant  incremental  predictors  of  the  criterion  measures.  It 
was  found,  as  shown  in  Table  2,  that  in  the  49  cases  where  at 
least  one  of  the  two  tests  was  a  significant  predictor,  there 
were  17  instances  where  both  measures  entered  the  prediction 
equation. 


DISCUSSION 

To  properly  interpret  the  results  above,  it  is  important  to 
note  that  there  is  a  certain  degree  of  redundancy  among  some 
criteria.  First  of  all,  written  and  hands-on  tests  contributed 
variance  to  either  the  Total  Score  on  School  and  Job  Knowledge 
Tests  or  the  Total  Score  on  Hands-On  Tests,  as  well  as  to  General 
Soldiering  and  Core  Technical  Proficiency.  Along  the  same  lines, 
some  of  the  more  specific  criterion  measures  (e.g..  Navigation, 
Target  Identification)  form  a  small  part  of  the  overall  success 
scores,  such  as  General  Soldiering  and  Core  Technical 
Proficiency.  Another  methodological  consideration  is  that  the 
Figural  Reasoning  and  Mazes  tests  are  in  a  paper-and-pencil 
format,  introducing  the  question  of  "method  variance"  into  their 
correlations  with  the  paper-and-pencil  criterion  measures 
(although  variance  due  to  verbal  ability  is  perhaps  only  a  small 
portion  of  total  variance  on  the  Figural  Reasoning  and  Mazes 
test) . 

The  cautions  above  notwithstanding,  the  results  of  the 
analyses  indicate  a  reasonably  high  probability  that  in  the 
future  the  Figural  Reasoning  test  will  be  shown  to  be  more  useful 
than  the  Mazes  test  for  incrementing  the  validity  of  ASVAB  for 
predicting  performance  criteria  across  a  wide  variety  of  MOS. 
These  results  are  directly  relevant  to  the  selection  of  tests  to 
be  used  in  the  inter-Service  Enhanced  Computer  Assisted  Testing 
(ECAT)  research. 
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Table  1 


fiiqnifican-h  Semi-part ial  Correlations  for  Fiaural  Reasoning 
and  Mazes  Tests  as  Incremental  Predictors  of  Criteria. 

MOS 

IIB®  13B  19E  31C  63B  64C  71L  91A  95B 


Total  Score  on  School  and  Job  Knowledge  Tests; 


Figural  Reas. 

Mazes 

.06  .07 

.06 

.07 

.07 

.05 

.05 

.08 

.07 

.07 

.07 

.06 

.06 

.06 

Total  Score  on 

Hands-On  Tests: 

Figural  Reas. 

.08 

.12 

.14 

.13 

.12 

.09 

.07 

Mazes 

.07 

r:onp.rai  Soldiering  Proficiencv: 

Figural  Reas. 

Mazes 

na  .  10 

na  .  12 

.14 

.14 

(none) 

.07 

.08 

.07 

,07 

Core  Technical 

Proficiencv: 

Figural  Reas. 

Mazes 

.07 

.07 

.07 

.09 

.09 

-.05 

-.08 

.13 

.13 

sifin  Onai  Ification  Test  Score: 

Figural  Reas. 

Mazes 

.08 

.10 

.08 

.07 

(none) 

.08 

.10 

na 

na 

Navigation: 
Figural  Reas. 

na  .  14 

na  .14 

.09 

.13 

Mazes  na 

na 

Target  Identification  f excludes  MOS  63B  and  71L) : 
Figural  Reas.  (none) 

Mazes  (none) 


Table  1  (cont.) 

significant  Semi-partial  Correlations  for  Fiaural  Reasoning 
and  Mazes  Tests  as  Incremental  Predictors  of  Criteria 


MOS 

IIB®  13B  19E  31C  63B  64C  71L  91A  95B 


oneratina  Howitzer  Sights  (MOS  13B  only) : 

Figural  Reas.  (none) 

Mazes  (none) 

Tank  Gunnery  (MOS  19E  only) ; 

Figural  Reas.  (none) 

Mazes  (none) 

Determining  Grid  Coordinates; 

Figural  Reas.  na  na  .14  .08  na  .09  .10  .07 

na  na  .14  na  .09  .10 

Mazes  (none) 


Note.  Semipartial  r's  for  significant  predictors  only,  separately 
for  two  orders  of  entry  (ASVAB-Spatial-Psychomotor  and  ASVAB- 
Psychomotor-Spatial) ;  "na"  is  shown  if  a  criterion  was  not  tested 
in  the  MOS. 

®  The  MOS  were  as  follows; 

IIB  -  Infantrymen  (N=491) 

13B  -  Cannon  Crewmen  (N=464) 

19E  -  Armor  Crewmen  (N=394) 

3 1C  -  Single  Channel  Radio  Operator  (N=289) 

63B  -  Light  Wheel  Vehicle  Mechanic  (N=478) 

64 C  -  Motor  Transport  Operator  (N=507) 

71L  -  Administrative  Specialist  (N=427) 

91A  -  Medical  Specialist  (N=392) 

95B  ^  Military  Police  (N=597) 
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Table  2 


wmnh^-r  of  MOS  for  which  Assembling  Objects  and  Fiqural  Reasoning 
are  Significant*  Predictors  of  Criteria 


Both 

Assembling 

Objects 

Figure 1 
Reasoning 

Neither 

Total 

School/ Job  Know  Tests: 

6 

2 

1 

0 

9 

Hands-On  Tests; 

1 

1 

3 

4 

9 

Gen  Soldiering  Prof; 

4 

3 

1 

0 

8 

Core  Technical  Prof: 

2 

3 

2 

2 

9 

Skill  Qual  Test; 

1 

4 

2 

1 

8 

Navigation: 

1 

3 

1 

3 

8 

Target  Identification; 

0 

1 

0 

6 

7 

Align  Howitzer  Sights; 

0 

1 

0 

0 

1 

Tank  Gunnery; 

0 

0 

0 

1 

1 

Deter  Grid  Coordinates; 

2 

1 

3 

0 

6 

TOTAL 

17 

19 

13 

17 

66 

®  In  either  or  both  orders  of  entry. 


8 


Working  Paper 


THE  USEFULNESS  OF  PROJECT  A  SPATIAL  TESTS 
FOR  PREDICTING  COMPREHENSIVE  PERFORMANCE  MEASURES 


Henry  H.  Busciglio 


Selection  and  Classification  Technical  Area 
Manpower  and  Personnel  Research  Laboratory 


U.  S.  Army  Research  Institute 
for  the  Behavioral  and  Social  Sciences 


5001  Eisenhower  Avenue 
Alexandria,  VA  22333-5600 


February,  1990 


The  Usefulness  of  Project  A  Spatial  Tests 
For  Predicting  Comprehensive  Performance  Measures 

Introduction 

The  Army's  Project  A  is  a  long-term,  comprehensive  effort  to 
improve  the  selection  and  classification  of  enlisted  personnel. 
One  objective  of  this  effort  was  to  validate  the  Armed  Services 
Vocational  Aptitude  Battery  (ASVAB) ,  the  traditional  instrument 
for  assessing  the  general  cognitive  skills  of  potential  soldiers. 
Previous  analyses  of  Project  A  data  (Campbell,  1988)  demonstrated 
that  the  ASVAB  is  useful  for  predicting  first  tour  performance, 
particularly  such  "can-do"  criteria  as  General  Soldiering 
Proficiency  and  Core  (i.e.,  MOS-specif ic)  Technical  Proficiency. 
Therefore,  the  ASVAB  serves  as  a  baseline  against  which  the 
marginal  utility  of  other  tests  for  selection  and  classification 
is  judged. 

Another  objective  of  Project  A  was  to  develop  and  validate 
measures  of  abilities  other  than  the  general  cognitive  domain 
covered  by  ASVAB.  For  example.  Project  A  staff  members 
hypothesized  that  measures  of  spatial  and  perceptual-psychomotor 
abilities  would  account  for  criterion  variance  which  was  not 
predictable  from  ASVAB  scores.  In  pursuit  of  this  objective,  a 
set  of  new  Project  A  predictors  was  administered  to  9,500 
soldiers  in  19  Military  Occupational  Specialties  (MOS)  in  the 
1985  Concurrent  Validation  phase. 

In  a  previous  analysis  of  the  1985  Concurrent  Validity  data, 
Busciglio  (in  preparation)  found  that  Project  A  test  scores 
substantially  improved  the  prediction  of  many  criteria,  most 
notably  three  comprehensive  measures  of  success:  total  score  on 
written  tests  of  school  and  job  knowledge.  General  Soldiering 
Proficiency  and  Core  (i.e.,  MOS-specific)  Technical  Proficiency. 
The  statistical  procedure  used  was  a  series  of  backward  stepwise 
regressions  in  which  the  ASVAB  subtests  were  entered  into  the 
equation  in  a  block  and  nonsignificant  subtests  were  removed, 
then  the  Project  A  tests  were  entered  to  determine  the  extent  to 
which  they  were  significant  as  incremental  predictors. 

The  present  analyses  are  similar  except  that  spatial  tests 
were  entered  first  and  then  the  ASVAB  subtests.  Two  research 
topics  were  addressed: 

1.  How  much  criterion  variance  can  the  spatial  tests  alone 
account  for,  and  how  does  this  compare  with  the  earlier  results 
for  ASVAB? 

2.  Which  individual  spatial  tests  will  remain  significant 
predictors  of  the  criterion  measures  after  the  ASVAB  subtests 
have  been  entered  into  the  prediction  equations? 
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Method 


Subjects 

Subjects  were  first-term  enlisted  personnel  in  the  nine  MOS 
for  which  hands-on  criterion  measures  were  collected  as  part  of 
the  1985  Concurrent  Validation  phase  of  Project  A.  These  MOS  had 
been  selected  for  comprehensive  examination  in  Project  A  because 
they  were  judged  to  be  representative  of  the  entire  population  of 
entry-level  Army  MOS.  The  sample  consists  of  individuals  who  had 
entered  the  Army  between  1  July  1983  and  30  July  1984  and  was 
drawn  from  thirteen  posts  in  the  continental  United  States  as 
well  as  the  U.S.  Army  in  Europe  (USAREUR) .  The  number  of 
subjects  from  each  MOS,  as  well  as  the  total  sample  size,  is 
shown  in  Table  1. 

Predictors 

Predictors  were  the  nine  ASVAB  subtests  and  the  six  Project 
A  paper-and-pencil  tests  of  spatial  ability.  Detailed 
information  concerning  the  nature  and  development  of  the  Project 
A  predictors  can  be  found  in  Peterson  (1987) .  Table  2  presents  a 
list  of  these  predictors. 

Criterion  Measures 

All  criteria  included  in  these  analyses  are  comprehensive, 
"can-do"  measures  of  proficiency  in  performing  job  duties,  as 
described  below. 

Total  Score  on  School  and  Job  Knowledge  Tests.  School  and 
job  knowledge  tests  are  written,  multiple-choice  measures  of 
soldiers'  technical  knowledge  pertinent  to  the  various  tasks 
performed  in  each  MOS.  Each  school  knowledge,  or  end-of -training 
test  consists  of  130-210  items,  depending  upon  MOS  (Davis,  Davis, 
Joyner,  &  de  Vera,  1987) .  The  job  knowledge  tests  are  meant  to 
measure  knowledge  of  between  25  and  31  "critical  tasks"  and 
consist  of  150-200  items,  once  again  depending  upon  the 
particular  MOS  (Campbell,  in  preparation).  Unlike  items  on  the 
school  knowledge  tests,  those  on  the  job  knowledge  tests  were 
selected  during  Project  A  to  sample  content  broadly  and  to 
discriminate  maximally  among  examinees.  The  total  score  is  a 
unit-weighted  composite  of  standard  scores  on  the  school  and  job 
knowledge  tests. 

General  Soldiering  Proficiency.  General  Soldiering  and  Core 
Technical  Proficiency  are  the  two  "performance  constructs"  which 
the  Project  A  staff  created  to  account  for  the  variance  in  "can- 
do"  performance.  General  Soldiering  Proficiency  is  a  composite 
score  on  a  variety  of  tasks  common  to  many  MOS  (e.g.,  determining 
grid  coordinates  on  maps,  recognizing  friendly  vs.  threat 
aircraft),  as  measured  by  written  test  items  and  hands-on  tasks. 
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That  is,  the  composite  is  an  aggregate  across  measurement 
methods.  (MOS  IIB  was  not  scored  on  this  measure.) 

Core  fi.e..  MOS-soecif ic^  Technical  Proficiency.  This  is 
defined  as  soldiers'  ability  to  perform  the  tasks  that  are  at  the 
"core"  of  each  MOS  (i.e.,  those  that  define  the  MOS),  the 
composite  score  being  an  aggregate  of  scores  on  written  test 
items  and  hands-on  tasks. 

Skill  Qualification  Test  Score  fSOT) .  These  are  paper-and- 
pencil  tests  of  MOS-specific  technical  knowledge  developed  by  the 
U.S.  Army  Training  and  Doctrine  Command  for  periodic  testing  of 
soldiers  in  their  MOS.  This  was  the  only  criterion  analyzed 
which  was  not  developed  in  Project  A.  (MOS  91A  was  not  scored  on 
this  measure.) 

It  should  be  stressed  that  the  comprehensive  measures  above 
are  not  mutually  exclusive.  That  is,  written  test  scores  were 
used  in  the  computation  of  General  Soldiering  and  Core  Technical 
Proficiency,  as  well  as  the  total  score  for  written  tests. 

Procedure 


Collection  of  Project  A  predictor  and  criterion  data  was 
part  of  the  1985  concurrent  validation  and  occurred  between  10 
June  and  13  November  of  that  year.  Scores  on  the  ASVAB  subtests 
and  the  Skill  Qualification  Test  were  obtained  from  archival  data 
sources. 

To  answer  the  questions  raised  in  the  introduction,  a  series 
of  backward  stepwise  multiple  regression  analyses  were  performed 
separately  for  each  MOS.  This  procedure  begins  by  entering  all 
specified  predictors  into  the  equation  as  a  block,  and  then 
removes  nonsignificant  predictors  one  by  one,  based  upon  their 
individual  contributions  to  the  overall  with  the  criterion. 
This  backward  elimination  procedure  continues  until  all  variables 
in  the  equation  are  individually  significant  at  a  certain 
probability  level  (the  present  analyses  used  .05). 

The  Army's  selection  and  classification  decisions  are  based 
on  the  ASVAB  subtests.  Because  of  this,  restriction  of  range  in 
the  ASVAB  scores  (as  well  as  any  implicit  range  restriction  in 
the  measures  which  are  correlated  with  them)  would  probably  lead 
to  underestimates  of  the  actual  criterion  variance  which  would 
have  been  accounted  for  if  ASVAB  scores  had  not  been  used  as  a 
selection  screen.  To  correct  for  this  problem,  matrices  of 
estimated  population  (i.e.,  unrestricted)  covariances  among 
predictors  and  criteria  were  created,  using  the  Lawley  formula 
presented  in  Lord  and  Novick  (1968;  pp.  184-188),  and  used  as 
input  in  these  analyses.  The  R^s  reported  below  are  in  terms  of 
this  unrestricted  population  (the  1980  youth  population,  composed 
of  individuals  between  the  ages  of  18  and  23)  and  have  also  been 
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adjusted  for  shrinkage,  using  the  formula  given  by  Wherry  (1940) . 

Employing  an  SPSS  Regression  program  with  both  the  ''Enter'' 
and  "Backward  Stepwise"  subcommands,  the  analyses  proceeded  in 
the  following  two  stages: 

1.  SPATIAL  - >  (SPATIAL) 

The  six  spatial  tests  were  entered  as  a  block  into  each  equation 
and  the  backward  procedure  determined  which  were  individually 
significant. 

2.  (SPATIAL) +ASVAB  - >  (SPATIAL+ASVAB) 

Significant  spatial  tests  were  retained  in  a  second  stage  of 
analysis,  meaning  that  they  were  no  longer  subject  to  removal. 
The  ASVAB  subtests  were  entered  as  a  block  and  retained  only  if 
they  added  individually  significant  variance  to  the  obtained 
for  the  spatial  tests  alone. 

The  results  of  the  present  analysis  were  compared  to  those 
obtained  by  Busciglio  (in  preparation),  which  included  the 
following  stepwise  procedures  to  assess  the  extent  to  which  the 
spatial  predictors  could  increment  ASVAB: 

1.  ASVAB  - >  (ASVAB) 

The  nine  ASVAB  subtests  were  entered  as  a  block  into  each 
equation  and  the  backward  procedure  determined  which  were 
individually  significant. 

2.  (ASVAB) +SPATIAL  - >  (ASVAB+SPATIAL) 

Significant  ASVAB  subtests  were  retained  in  a  second  stage  of 
analysis,  meaning  that  they  were  no  longer  subject  to  removal. 
The  spatial  tests  were  entered  as  a  block  and  retained  only  if 
they  added  individually  significant  variance  to  the  R^  obtained 
for  ASVAB  alone. 


Results 


First  Research  Topic 

Table  3  shows  R^s  and  incremental  R^s  for  spatial  and  ASVAB 
tests  as  groups,  comparing  the  results  of  the  present  analyses 
with  those  of  the  earlier  investigation  (Busciglio,  in 
preparation) .  As  can  be  seen,  in  many  of  the  comparisons  the 
spatial  tests  account  for  approximately  as  much  criterion 
variance  at  Stage  1  of  the  analyses  as  do  the  ASVAB  tests.  The 
exceptions  are  shown  in  Table  4,  which  lists  the  criterion 
measures  and  MOS  where  the  difference  in  R^s  for  the  two  groups 
of  tests  equals  or  exceeds  .05  (an  arbitrary  cutoff).  As  the 
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table  shows,  the  ASVAB  tests  were  superior  to  the  spatial 
measures  in  all  these  instances,  and  were  clearly  better 
predictors  of  the  two  written  criteria  -  Total  Score  on  School 
and  Job  Knowledge  Tests  and  the  Skill  Qualification  Test.  In 
contrast,  the  superiority  of  the  ASVAB  tests  for  predicting 
General  Soldiering  and  Core  Technical  Proficiency,  which  are 
composites  of  written  and  hands-on  measures,  was  limited  to  a 
smaller  number  of  MOS. 

There  was  some  evidence  that  the  superiority  of  ASVAB  over 
spatial  tests  may  be  a  function  of  MOS.  As  Table  4  indicates, 
the  ASVAB  subtests  were  superior  predictors  of  most  of  the 
criterion  measures  in  three  MOS:  31C,  63B,  and  91A. 

Second  Research  Topic 

Table  5  lists  the  individual  spatial  tests  which  attained 
significance  in  Stage  1  and  notes  with  an  asterisk  those  which 
remained  significant  after  the  ASVAB  tests  were  entered  (and 
nonsignificant  ones  removed)  in  Stage  2.  The  summary  of  these 
results  below  shows  the  number  of  times  (out  of  a  maximum  of  34) 
that  each  spatial  test  was  significant  in  Stage  1  and  remained 
significant  in  Stage  2: 


Spatial  Test 

Stage 

1  Stage  2 

Assembling  Objects 

23 

21 

Figural  Reasoning 

27 

20 

Map 

34 

18 

Maze 

6 

0 

Object  Rotation 

5 

2 

Orientation 

16 

1 

seen.  Assembling  Objects, 

Figural  Reasoning 

were  especially  strong  predictors  of  the  criterion  measures 
across  MOS.  Assembling  Objects  and  Figural  Reasoning  remained 
significant  in  the  vast  majority  of  cases.  The  Map  test,  while 
the  strongest  of  all  predictors  in  Stage  1,  remained  significant 
a  smaller  percentage  of  the  time.  The  other  three  spatial  tests 
were  much  weaker  overall  and  remained  significant  in  only  a  very 
few  cases. 


Discussion 

Findings  pertinent  to  the  first  research  topic  indicated 
that  the  spatial  tests  measure  abilities  which  are  important 
across  a  wide  variety  of  MOS  and  criteria,  being  comparable  to 
ASVAB  as  predictors  of  the  two  composite  criteria.  Although  the 
spatial  tests  were  somewhat  inferior  as  predictors  of  the  two 
written  performance  measures,  method  variance  might  be 
responsible  for  these  differences  since  the  spatial  tests,  while 
in  a  written  format,  may  not  tap  as  much  verbal  or  "scholastic" 
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ability  as  the  ASVAB  tests. 

As  reported  above,  there  was  some  evidence  to  support  the 
contention  that  the  superiority  of  ASVAB  over  spatial  tests  for 
predicting  comprehensive  measures  of  performance  may  also  be  a 
function  of  MOS.  The  three  enlisted  jobs  in  the  present  sample 
where  this  seems  to  be  the  case  -  31C,  63B,  and  91A  -  are  not 
easily  distinguished  from  the  others  (see,  for  example,  the 
titles  of  these  jobs  in  Table  1).  Thus,  further  research  on  the 
Project  A  predictors  will  be  needed  in  order  to  determine  what, 
if  any,  characteristics  determine  the  extent  to  which  job 
performance  is  differentially  predictable  from  ASVAB  vs.  spatial 
measures . 

Referring  to  the  second  research  topic,  three  of  the  spatial 
tests  -  Assembling  Objects,  Figural  Reasoning,  and  Map  were  very 
strong  predictors  across  MOS  and  criteria  and  tended  to  remain 
significant  after  the  ASVAB  tests  were  entered  and  deleted  in 
Stage  2 .  This  supports  the  notion  that  these  tests  would  lead  to 
an  overall  improvement  in  the  prediction  of  job  performance  if 
incorporated  into  the  Army's  selection  composites.  On  the  other 
hand,  the  usefulness  of  the  Maze,  Object  Rotation,  and 
Orientation  tests  seems  to  be  much  more  limited  to  specific  MOS 
and/or  performance  measure.  Thus,  their  greatest  utility  would 
probably  be  in  MOS-specific  classification  composites. 

To  interpret  these  results  properly,  it  is  important  to  note 
a  number  of  methodological  considerations.  As  stated  earlier, 
individual  written  tests  contributed  variance  to  the  total 
written  score  on  school  and  job  knowledge  tests  as  well  as  to 
General  Soldiering  and  Core  Technical  Proficiency.  Also, 
possible  differences  in  motivation  due  to  differences  in  testing 
situations  (i.e.,  ASVAB  scores  used  for  selection  as  opposed  to 
Project  A  scores  used  "for  research  purposes  only")  may  have 
impacted  the  results.  That  is,  individuals  may  have  responded 
more  carefully,  exerted  more  effort,  etc.,  on  the  ASVAB  subtests, 
thus  making  them  more  valid  measures  of  abilities  than  the 
Project  A  tests.  The  final  methodological  concern  has  to  do  with 
the  statistical  analyses  used.  Stepwise  regression  procedures, 
while  useful  for  empirically  exploring  alternative  models,  are 
especially  susceptible  to  sampling  error  (cf.  Cohen  &  Cohen, 

1983) .  The  samples  used  in  the  analyses  were  generally  of 
sufficient  size  to  make  the  degree  of  shrinkage  in  each 
individual  equation  relatively  low,  but  the  large  number  of 
equations  computed  here  increases  the  probabilities  that  some 
ASVAB  and  Project  A  predictors  are  significant  due  to  Type  I 
errors.  However,  it  should  be  noted  that  each  spatial  predictor 
can  be  significant  a  maximum  of  34  times,  across  all  criteria  and 
MOS.  At  an  alpha  level  of  .05,  a  predictor  may  be  expected  to 
reach  significance,  by  sampling  error  alone,  between  one  and  two 
times.  As  Table  5  clearly  indicates,  most  of  the  Project  A  tests 
were  significant  far  more  often  than  this.  Nevertheless,  the 


6 


lack  of  opportunities  at  this  point  for  cross-validation  renders 
the  results  reported  in  this  paper  suggestive  only. 
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Table  1 


Subjects  for  the  Present  Analysis 


MOS 

Enlisted  Job 

N 

SQT 

IIB 

Infantry 

491 

444 

13B 

Cannon  Crew 

464 

396 

19E 

Armor  Crew 

394 

338 

3 1C 

Single  Channel  Radio  Operator 

289 

248 

63B 

Light  Wheel  Vehicle  Mechanic 

478 

409 

64C  (now  88M) 

Motor  Transport  Operator 

507 

427 

71L 

Administrative  Specialist 
Medical  Specialist 

427 

361 

91A 

392 

0 

95B 

Military  Police 

597 

545 

TOTAL 

4,039 

3,168 

Note.  Sample  sizes  shown  in  SQT  column  were  those  available 
for  the  analysis  of  the  Skill  Qualification  Test  criteria, 
as  described  later  in  this  paper. 
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Table  2 


Predictors  Used  in  the  Analysis 


ASVAB  Subtests: 

Arithmetic  Reasoning 
Auto/Shop  Information 
Coding  Speed 
Electronics  Information 
General  Science 
Math  Knowledge 
Mechanical  Comprehension 
Number  Operations 
Verbal  (Paragraph  Comprehension 
+  Word  Knowledge) 


Spatial  Ability  Tests: 
Assembling  Objects 
Figural  Reasoning 
Map 
Maze 

Object  Rotation 
Orientation 
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Table  3 


Comparison 

of  R^s  for  Spatial  and  ASVAB 

Tests 

Analysis 

Present 

Previous 

Stage 

(1)  (2) 

(1)  (2) 

Predictors 

(SPATIAL)  (SPATIAL+ASVAB) 

(ASVAB)  (ASVAB+SPATIAL) 

Total  Score  on 

School 

and  Job  Knowledae 

Tests : 

IIB 

.59 

.66 

.59 

.66 

13B 

.40 

.45 

.39 

.45 

19E 

.57 

.64 

.58 

.64 

3 1C 

.50 

.61 

.60 

.62 

63B 

.50 

.66 

.63 

.66 

64C 

.49 

.59 

.55 

.59 

71L 

.55 

.61 

.54 

.61 

91A 

.54 

.70 

.67 

.69 

95B 

.58 

.68 

.62 

.68 

MEDIAN 

.54 

.64 

.59 

.64 

General  Soldiering  Proficiency; 


IIB 

— 

— 

— 

— 

13B 

.30 

.33 

.30 

.33 

19E 

.47 

.50 

.44 

.50 

31C 

.45 

.51 

.49 

.52 

63B 

.31 

.33 

.28 

.33 

64C 

.46 

.53 

.49 

.53 

71L 

.44 

.48 

.41 

.48 

91A 

.49 

.56 

.54 

.56 

95B 

.56 

.64 

.59 

.64 

MEDIAN 

.46 

.51 

.47 

.51 

Core  Technical 
IIB 

Proficiency: 

.48 

.53 

.48 

.53 

13B 

.16 

.18 

.15 

.18 

19E 

.32 

.39 

.35 

.38 

3 1C 

.38 

.54 

.54 

.54 

63B 

.32 

.50 

.48 

.50 

64C 

.28 

.35 

.32 

.36 

71L 

.42 

.49 

.44 

.49 

91A 

.45 

.59 

.58 

.59 

95B 

.39 

.45 

.43 

.46 

MEDIAN 

.38 

.49 

.44 

.49 
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Table  3  (cont.) 


Comparison 

of  R^s  for  Spatial  and  ASVAB 

Tests 

Analysis 

Present 

Previous 

Stage 

(1) 

(2) 

(1) 

(2) 

Predictors 

(SPATIAL)  (SPATIAL+ASVAB) 

(ASVAB) 

(ASVAB+SPATIAL) 

Skill  Qualification 
IIB  .38 

Test  Score; 

.47 

.44 

.46 

13B 

.19 

.22 

.19 

.22 

19E 

.39 

.44 

.40 

.44 

31C 

.44 

.56 

.55 

.57 

63B 

.40 

.56 

.55 

.56 

64C 

.38 

.51 

.50 

.51 

71L 

.43 

.59 

.58 

.60 

91A 

— 

— 

— 

— — 

95B 

.45 

.60 

.60 

.61 

MEDIAN 

.40 

.54 

.53 

.54 
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Table  4 


Largest  Differences  in  R^s  for  Spatial  and  ASVAB  Tests 


Analysis  Present  Previous 

Stage  (1)  (2)  (1)  (2) 

Predictors  SPATIAL  (SPATIAL) +ASVAB  ASVAB  (ASVAB) +SPATIAL 


Total  Score  on 

School  and 

Job  Knowledoe 

Tests: 

31C 

.50 

.61 

.60 

.62 

63B 

.50 

.66 

.63 

.66 

64C 

.49 

.59 

.55 

.59 

91A 

.54 

.70 

.67 

.69 

MEDIAN 

.54 

.64 

.59 

.64 

General  Soldierina  Proficiency: 

91A 

.49 

.56 

.54 

.56 

Core  Technical 

Proficiency: 

3 1C 

.38 

.54 

.54 

.54 

63B 

.32 

.50 

.48 

.50 

91A 

.45 

.59 

.58 

.59 

MEDIAN 

.38 

.49 

.44 

.49 

Skill  Qualification  Test 

Score : 

IIB 

.38 

.47 

.44 

.46 

31C 

.44 

.56 

.55 

.57 

63B 

.40 

.56 

.55 

.56 

64C 

.38 

.51 

.50 

.51 

71L 

.43 

.59 

.58 

.60 

95B 

.45 

.60 

.60 

.61 

MEDIAN 

.40 

.54 

.53 

.54 

Note.  Differences  between  R^s  for  Spatial  and  ASVAB  are  greater 
than  or  equal  to  .05  (an  arbitrary  cutoff). 
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Table  5 


Significant  Spatial  Predictors  at  Stage  1  of  Analyses 


Total  Score  on  School  and  Job  Knowledge  Tests; 


IIB 

Fig  Reas* 

Map* 

Orient 

13B 

Assem  Objs* 

Map* 

Obj 

Rota 

19E 

Assem  Objs* 

Fig  Reas* 

Map* 

3 1C 

Fig  Reas* 

Map 

Maze 

63B 

Assem  Objs* 

Fig  Reas* 

Map* 

Orient 

64C 

Assem  Objs* 

Fig  Reas* 

Map 

71L 

Assem  Objs* 

Fig  Reas* 

Map* 

91A 

Assem  Objs* 

Fig  Reas* 

Map 

95B 

Assem  Objs* 

Fig  Reas* 

Map* 

Orient 

General  Soldierina 

Proficiency 

I 

13B 

Assem  Objs* 

Map* 

Maze 

19E 

Assem  Objs* 

Fig  Reas* 

Map* 

31C 

Fig  Reas* 

Map 

Maze 

63B 

Assem  Objs* 

Map* 

Orient 

64C 

Assem  Objs* 

Fig  Reas* 

Map 

Obj 

Rota* 

Orient* 

71L 

Assem  Objs* 

Fig  Reas 

Map 

Obj 

Rota 

Orient 

91A 

Assem  Objs* 

Fig  Reas* 

Map 

Orient 

95B 

Assem  Objs* 

Fig  Reas* 

Map* 

Orient 

Core 

Technical  Proficiency: 

IIB 

Fig  Reas* 

Map* 

Orient 

13B 

Assem  Objs* 

Map* 

19E 

Assem  Objs* 

Map* 

Obj 

Rota* 

31C 

Fig  Reas 

Map 

Maze 

63B 

Assem  Objs* 

Map 

Orient 

64C 

Assem  Objs* 

Fig  Reas* 

Map 

71L 

Assem  Objs* 

Fig  Reas* 

Map* 

Obj 

Rota 

91A 

Assem  Objs 

Fig  Reas* 

Map 

95B 

Fig  Reas* 

Map* 

Orient 

Skill 

Oualif ication  Test  Score 

« 

• 

IIB 

Fig  Reas* 

Map* 

Orient 

13B 

Assem  Objs* 

Map* 

19E 

Assem  Objs* 

Fig  Reas 

Map* 

3 1C 

Fig  Reas 

Map 

Maze 

Orient 

63B 

Assem  Objs 

Fig  Reas 

Map 

Orient 

64C 

Fig  Reas 

Map 

Maze 

Orient 

71L 

Fig  Reas* 

Map 

95B 

Fig  Reas 

Map 

Orient 

*  Remained  significant  after  entry  of  ASVAB  tests  in  Stage  2. 
Assem  Objs  -  Assembling  Objects  Obj  Rota  -  Object  Rotation 

Fig  Reas  -  Figural  Reasoning  Orient  -  Orientation 
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An  Outlln*  of  Issues  Pertaining  to  EEO  Concerns  and  Project  A 

John  P.  Campbell 

Since  the  passage  of  the  1964  Civil  Rights  Act  and  the  beginning  of  litiga¬ 
tion  pertaining  to  discrimination  in  personnel  selection,  a  great  deal  of 
effort  has  focused  on  carrying  out  research  on  the  nature  of  employment 
discrimination,  the  explication  of  its  meaning,  and  the  development  of 
potential  remedies.  There  now  is  a  very  large  literature  on  the  topic, 
both  conceptual  and  empirical,  and  virtually  every  professional  doing  busi¬ 
ness  in  personnel  research  or  personnel  management  is  familiar  with  it. 

The  purpose  of  the  memo  is  simply  to  remind  everyone  on  Project  A  of  the 
issues  Involved  and  to  cite  the  relevant  documentation  and  professional 
consensus,  when  it  exists. 

What  follows  is  organized  into  three  sections.  First,  some  basic  terms  and 
their  definitions  are  summarized.  Second,  the  major -questions  and  issues 
that  comprise  this  topic  are  briefly  outlined.  For  each  issue  relevant 
literature  sources  will  be  cited  along  with  whatever  professional  consensus 
exists  as  to  appropriate  conclusions  that  can  be  drawn.  The  third  section 
will  attempt  to  summarize  the  relevance  of  these  issues  for  Project  A.  The 
definitive  treatment  of  these  issues  is  to  be  found  in  the  two  volume 
report  on  "Ability  Test:  Uses,  Consequences,  and  Controversies"  published 
by  ttie  National  Research  Council  and  written  by  the  Committee  on  Ability 
Testing,  National  Academy  of  Sciences  (Wigdor  &  Garner,  1982). 


SOME  DEFINITIOKS 


The  context  for  this  discussion  is  that  of  personnel  decision  making;  and 
more  specifically  a)  selection  and  classification  decisions  for  new 
employees  and  b)  promotion  decisions  for  incumbent  employees.  These  two 
decisions  are  at  the  center  of  most  of  the  literature  on  the  topic.  Each 
decision  implies  there  is  some  available  information  (e.g.,  an  interview  or 
psychological  test)  upon  which  the  decision  is  made  and  some  means  by  which 
the  accuracy  or  validity  of  the  decision  can  be  assessed. 

Given  this  context,  the  major  terms  and  concepts  on  which  to  be  clear  are 
the  following. 

VALIDITY 

A  professional  consensus  concerning  the  appropriate  definition  of  validity 
is  reflected  in  the  professional  literature  and  the  policy  statements  of 
the  American  Psychological  Association,  the  American  Council  on  Measurement 
in  Education,  and  the  American  Educational  Research  Association.  In  the 
context  of  personnel  decision  making  the  consensus  is  that  a  selection  or 
promotion  standard  is  valid  if  there  is  significant  evidence  to  support  the 
specific  use  for  which  it  is  intended.  Significant  evidence  may  be 
accumulated  in  several  ways  and  the  concepts  of  criterion  related 
validation,  content  validation,  and  construct  validation  still  constitute 
the  most  appropriate  and  professionally  acceptable  description  of 
validation  strategies.  Consequently,  while  users  are  obligated  to  justify 
the  use  of  particular  decision  making  information  with  evidence,  no  one 
particular  kind  of  evidence  can  be  judged  better  than  another  on  i  priori 
grounds.  The  evidence  that  should  be  used  depends  upon  the  nature  of  the 
decision  to  be  made,  the  context  in  which  it  must  be  made,  the  state  of 
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previous  research,  and  the  feasibility  of  gathering  various  kinds  of 
evidence.  The  definitive  statement  on  these  issues  is  the  forthcoming 
"Joint  Standards  for  the  Development  and  Use  of  Psychological  Tests"  to  be 
published  by  APA,  ACME,  and  AERA  in  1985. 

DISCRIMINATION  (IN  SELECTION  AND  aASSIFICATION) 

Taken  literally,  the  term  simply  refers  to  the  act  of  making  distinctions 
among  job  candidates.  Consequently,  discrimination  could  be  carried  out  by 
choosing  job  candidates  randomly,  or  on  the  basis  of  job  qualifications,  or 
on  the  basis  of  skin  color,  gender,  hairlength,  family  background,  etc.. 
There  are  hundreds  of  different  ways  that  applicants  could  be  hired  and 
classified.  Some  of  these  methods  are  specifically  forbidden  by  law. 

Others  are  deemed  desirable  or  undesirable  on  other  grounds.  For  example, 
hiring  people  by  random  selection  from  the  available  applicants  is  not 
forbidden  by  any  current  or  previous  statute.  However,  virtually  everyone 
agrees  that  organizations  should  not  have  to  use  random  selection  if  they 
don't  want  to.  The  notion  that  people  should  be  selected  (discriminated) 
on  the  basis  of  merit  is  still  prevailing  value  judgment  in  U.S.  society. 

Much  of  the  current  consensus  as  to  what  kinds  of  discrimination  are 
desirable  and/or  legal  and  which  are  not,  has  built  up  during  the  20  years 
since  the  passage  of  the  Civil  Rights  Act  of  1964.  Professional  research, 
legislation,  and  litigation  of  individual  cases  during  that  period  have 
contributed  to  this  consensus.  Developments  during  this  20  year  period  are 
thoroughly  documented  and  described  in  other  sources  and  will  not  be 
repeated  here.  The  important  points  for  our  situation  are  that. selection 
decisions  (i.e.,  a  discrimination  between  those  that  are  hired  and  those 
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that  are  not  hired)  must  be  based  on  standards  (i.e.,  predictors)  that 
reflect  legitimate  job  qualifications  (i.e.,  merit).  A  job  standard  is 
legitimate  if  it  is  a  valid  predictor  of  job  behavior,  performance,  or 
outcomes  that  represent  a  business  necessity.  This  is  another  way  of 
saying  that  selection  measures  (e.g.,  tests,  interviews,  job  histories, 
educational  histories,  etc.)  can  be  used  to  discriminate  among  applicants 
if  they  have  significant  validity  for  predicting  a  criterion  that  is 
relevant  for  the  accomplishment  of  organizational  goals.  All  legislation 
and  all  case  law  accept  the  principal  of  hiring  on  the  basis  of  merit. 
Discrimination  in  hiring  on  the  basis  of  race,  religion,  gender,  ethnic 
background,  or  national  origin  is  forbidden  by  Title  YII.  Discrimination 
on  the  basis  of  other  variables  unrelated  to  business  necessity  is  also 
grounds  for  legal  action  on  behalf  of  the  individual  who  is  denied 
employoment.  Such  actions  can  be  taken  under  a  variety  of  legal 
protections  (e.g.,  State  fair  employment  practice  laws.  Article  XIY  of  the 
U.S.  Constitution).  The  central  issue  is  still  whether  or  not  the 
selection  standard  is  a  valid  reflection  of  business  necessity. 

BIAS 

The  term  bias  is  widely  used  with  a  multiplicity  of  meanings,  and  is 
difficult  to  define  in  any  systematic  way.  Perhaps  the  best  professional 
reference  on  the  topic  is  the  article  by  Cole  (1981).  What  is  presented 
here  is  simply  an  outline  of  her  major  distinctions.  In  general,  the  term 
refers  to  the  fact  that  scores  on  a  measure  are  influenced  systematically 
by  factors  that  are  irrelevant  for  the  stated  purpose  of  measurement.  The 
factors  of  specific  interest  for  us  are  membership  vs.  non  membership  in 
one  of  the  classes  protected  by  Title  YII.  Within  this  context  there  are 
at  least  the  following  kinds  of  potential  bias. 
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Selection  Bias  -  Occurs  when  the  selection  decision  itself  systematically 


reflects  class  membership  but  when  such  class  Identification  has  nothing  to 
do  with  job  qualifications. 

Test  Bias  -  Refers  to  systematic  group  differences  in  total  test  scores 
that  are  irrelevant  or  unrelated  to  the  purpose  of  measurement  (e.g.,  group 
differences  on  a  test  of  perceptual  speed  and  accuracy  that  have  nothing  to 
do  with  the  construct  the  test  is  intended  to  measure).  Test  bias  could 
also  occur  even  though  group  means  were  similar  if  the  properties  measured 
by  the  test  were  substantially  different  for  the  two  groups. 

Criterion  Bias  -  Refers  to  systematic  differences  between  groups  that  are 
unrelated  to  the  components  of  job  performance  that  the  criterion  is 
intended  to  measure. 

DIFFERENTIAL  PREDICTION 

Differential  prediction  has  a  specific  psychometric  meaning  and  refers  to  a 
significant  difference  between  the  prediction  functions  for  two  groups  when 
the  task  is  to  predict  a  measure  of  job  performance  from  a  selection 
measure.  The  function  most  often  compared  is  the  linear  regression  of  the 
criterion  on  the  predictor.  Such  functions  can  differ  in  terms  of  their 
slopes,  intercepts,  or  standard  errors  of  measurement.  If  the  data  are 
available,  most  professionals  much  refer  to  compare  the  complete  regression 
functions  rather  than  the  zero  order  correlation  coefficients.  A 
significant  difference  between  predictor/criterion  correlations  for  the  two 
groups  is  usually  termed  differential  validity. 
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ADVERSE  IHPACT 

This  is  a  legal  term  that  refers  to  differences  in  selection  ratios,  or 
hiring  rates,  across  groups  when  they  are  candidates  for  the  same  job. 

While  the  numerator  in  the  two  ratios  is  the  number  of  people  hired  in  each 
group,  it  is  not  always  clear  what  the  denominators  should  be.  Are  they  a 
function  of  the  actual  applicant  pool,  a  potential  applicant  pool,  or  the 
labor  force  itself?  Typically,  organizations  have  not  been  required  by  the 
Courts  to  meet  unreasonable  demands  regarding  their  applicant  pools. 

FAIRNESS 

The  term  fairness,  or  more  specifically  selection  fairness,  is  also  not  a 
psychometric  term  but  refers  to  whether  or  not  the  individual  decisons  made 
by  the  selection  system  are  “fair”  when  held  up  against  some  set  of 
community  or  societal  standards.  Previous  attempts  to  give  a  psychometric 
definition  to  fairness  were  found  wanting  principally  because  the  models 
which  were  proposed  did  not  incorporate  the  necessary  value  judgments 
(Peterson  &  Novick,  1976). 

QUESTIONS,  ISSUES  AND  RESEARCH  FINDINGS 
Efforts  to  promote  equal  employment  opportunity  must  deal  systematically 
with  a  number  of  difficult  questions  and  issues.  For  some  of  these 
questions  and  issues  there  is  a  reasonably  large  research  literature  upon 
which  to  base  a  course  of  action.  For  others  there  is  not  and  much  more 
work  is  needed.  Listed  below  are  the  major  questions  and  issues  that  are 
of  current  concern  and  a  brief  summary  of  the  documentation  that  exists. 
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MEAN  DIFFERENCES 

As  documented  by  the  National  Academy  of  Sciences  panel  on  ability  testing, 
the  DoD's  Survey  of  American  Youth,  and  as  well  as  many  other  sources 
(e.g.,  Oensen,  1980),  there  are  significant  mean  difference  on  several  key 
variables  in  the  personnel  decision  making  context.  Probably  the  most 
consistent  differences  is  the  .75  “  1.0  standard  deviation  difference 

I 

frequently  observed  between  blacks  and  whites  on  standardized  tests  of 
verbal  and  arithmetic  ability,  with  the  white  mean  being  higher  than  the 
black  mean.  Mean  differences  are  frequently  observed  among  other  minority 
groups  as  well  with  the  mean  for  the  majority  group  usually  being  the 
highest.  There  are  far  fewer  differences  on  ability  tests  between  men  and 
women;  however,  differences  in  favor  of  males  are  frequently  found  on 
measures  of  spatial  relations  (Sevy,  1983).  Smaller  differences  are 
sometimes  observed  in  favor  of  females  on  tests  of  verbal  ability  and  in 
favor  of  males  on  arithmetic  and  mathematical  ability. 

The  consistent  mean  differences  that  are  found  for  majority-minority 
comparisons  frequently  do  produce  adverse  impact  for  minority  groups.  The 
mean  differences  are  not  a  product  of  unreliable  measurement  or  flawed  data 
collection  techniques.  However,  the  fact  that  they  exist  says  nothing 
about  their  origin.  The  hundreds  of  studies  that  have  been  done  do  not 
permit  any  choice  between  an  environment  or  heredity  explanation.  Another 
complication  is  that  when  the  comparisons  are  made  in  a  specific  seclection 
situation  the  two  groups  are  probably  not  sampled  in  the  same  way  from 
their  own  reference  population.  For  example,  the  individual  expectations 
and  perceptions  that  govern  self  selection  may  differ  for  the  two  groups 
and  thus  produce  applicant  pools  that  are  different.  Organizational 


7 


.recruitment  practices  may  also  differ  for  the  two  groups  as  when  the 
organization  tries  to  implement  affirmative  action  policies. 

DIFFEREHTIAL  PREDICTION 

There  is  now  a  large  research  literature  related  to  differential  prediction 
across  racial  and  gender  groups  in  both  employment  and  educational 
settings.  Again,  the  issue  is  whether  there  is  a  statistically  significant 
difference  in  linear  regression  lines  between  groups.  The  research 
results,  as  documented  in  the  National  Academy  of  Sciences  report,  are 
consistent  in  showing  that  in  both  employment  and  educational  settings 
there  is  seldom  any  significant  differences  in  prediction  functions  between 
blacks  and  whites  when  an  ability  test  is  used  to  predict  job  performance. 
Further,  the  differences  that  do  exist  virtually  always  result  in  the  job 
performance  of  blacks  being  overpredicted  when  a  conmon  regression  line  is 
used.  This  is  because  the  regression  line  for  blacks  is  almost  always 
below  that  for  whites.  Consequently,  it  would  virtually  never  be  in  the 
interests  of  the  minority  group  to  have  a  separate  prediction  equation. 

How  can  there  be  both  adverse  impact  for  blacks  vs.  whites  and  no 
significant  differential  prediction?  It  is  because  there  are  also  mean 
differences  on  the  criterion  and  they  are  about  what  would  be  predicted 
from  the  mean  differences  on  the  predictor. 

The  above  results  could  occur  because  both  tests  and  criteria  are  baised  or 
because  both  are  correctly  reflecting  an  average  deficit  in  black 
performance.  If  in  fact  job  performance  criteria  are  generally  unbiased, 
then  the  available  research  argues  that  selection  tests  are  also  unbiased. 
The  relative  mean  differences  are  about  what  would  be  expected  because  of 
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regression  effects.  Since  the  research  results  comparing  mean  differences 
are  reasonably  consistent  across  different  types  of  criteria,  which  at 
least  appear  to  have  varying  susceptibility  to  bias  (e.g.,  ratings  vs. 
performance  records),  there  is  a  stronger  empirical  argument  for  a  lack  of 
bias  in  performance  criteria  than  there  is  to  support  its  existence.  After 
carefully  reviewing  all  the  evidence,  the  NAS  panel  concluded  that  well 
developed  employment  tests  measuring  general  mental  abilities  are  not 
biased  against  blacks. 

These  data  do  not,  and  should  not,  preclude  the  development  of  additional 
selection  information  that  is  also  useful  for  predicting  various  aspects  of 
job  performance  and  which  does  not  show  adverse  impact.  For  example,  using 
tests  of  psychomotor  ability  to  predict  job  proficiency  or  using  interest/ 
biographical  information  to  predict  attrition  were  not  addressed  in  the 
research  examined  by  the  NAS  panel. 

In  contrast  to  the  research  on  black/white  differences,  there  is  much  less 
data  regarding  differential  prediction  across  gender  groups  and  the  results 
are  not  so  consistent.  As  yet,  we  have  few  consistent  findings  as  to 
whether  there  are  mean  differences  between  males  and  females  on  predictors 
such  as  mathematical  aptitude  or  spatial  relations  aptitude  that  are  not 
reflected  in  relevant  criterion  differences. 

A  complicating  factor  in  the  interpretation  of  group  differences  in 
regression  lines  is  that  under  certain  conditions  measurement  error  alone 
may  produce  such  differences  (Linn,  1984).  For  example,  if  the  regression 
of  the  criterion  true  scores  or  the  predictor  true  scores  is  the  same  for 
two  groups,  and  the  groups  do  indeed  differ  on  the  latent  variable{s)  that 
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produce  the  predictor/criterion  correlation  then  measuring  the  predictor 
and  criterion  with  less  than  perfectly  reliable  instruments  will  by  itself 
produce  a  difference  in  intercepts.  Consequently,  it  would  be  dangerous  to 
interpret  the  over  prediction  of  blacks  as  bias  against  whites. 

TEST  BIAS 

The  question  of  whether  gender  or  minority  group  membership  accounts  for 
variance  in  total  test  scores  that  is  independent  of  the  characteristics 
that  the  test  is  intended  to  measure  probably  cannot  be  answered 
definitely.  To  do  so  would  be  analogous  to  saying  that  the  construct 
validity  of  the  test  is  completely  determined,  or  that  every  possible 
alternative  explanation  has  been  determined.  Measurement  of  individual 
abilities  or  behavior  is  not  perfect,  regardless  of  whether  it  is  done  by 
psychologists,  economists,  teachers,  lawyers,  blacks,  or  whites. 

A  more  reasonable  question  is  whether  a  test  is  biased  in  the  way  that  it 
is  used.  We  are  concerned  with  the  use  of  tests  for  selection  and 
classification.  Are  tests  biased  for  that  purpose?  If  the  comparison  is 
between  blacks  and  whites  and  the  tests  in  question  are  well  developed 
standardized  tests  of  general  cognitive  abilities,  the  conclusion  reached 
by  NAS  panel  is  that  they  are  not.  Their  reasoning  and  the  literature  on 
which  it  is  based  is  very  carefully  described  in  Volume  I  of  their  report. 
Since  the  question  of  whether  there  is  test  bias  in  selection  is  virtually 
the  same  as  whether  there  is  differential  prediction  of  job  performance 
criteria  across  groups  the  same  evidence  applies  and  the  conclusions  are 
the  same . 
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ITEM  BIAS 


There  has  also  been  considerable  research  examining  whether  or  not  there  is 
an  interaction  between  item  content  or  item  type  and  group  membership 
(e.g.,  Carroll  and  Horn,  1981;  Jensen,  1980).  While  there  are  isolated 
examples  of  such  interactions,  there  has  been  no  general  finding  of  item  x 
group  interactions  (Carroll  and  Horn,  1981)  within  the  broad  array  of  items 
that  are  found  on  tests  of  general  cognitive  abilities.  Consequently,  it 
is  not  possible  to  attribute  the  source  of  the  group  differences  to 
particular  types  of  items.  Another  conclusion  from  these  data  is  that  the 
tests  are  apparently  measuring  the  same  construct  within  each  group. 

LINEAR  YS,  CURVILINEAR  PREDICTION 

An  issue  that  is  sometimes  raised  regarding  the  relationship  of  ability 
test  scores  to  job  performance  is  whether  or  not  the  regression  of 
performance  on  ability  is  actually  linear.  An  alternative  hypothesis  is 
that  after  a  certain  minimum  level  of  ability  is  reached  further  increments 
in  test  scores  do  not  mean  further  increments  in  job  performance.  Another 
way  of  stating  it  is  to  assert  that  beyond  a  certain  point  on  the  predictor 
continuum  people  can  be  selected  at  random  and  the  average  job  performance 
for  those  selected  would  not  be  significantly  lower  than  the  performance  of 
a  group  selected  from  the  "top  down"  on  the  predictor.  A  related  argument 
would  be  to  assert  that  once  an  individual  successfully  completes  a 
particular  course  of  training  or  instruction,  he  or  she  is  "qualified"  for 
a  particular  job  and  no  further  distinctions  need  be  made. 

The  evidence  on  this  issue  does  not  support  the  non-linear  argument.  For 
virtually  all  jobs  the  higher  the  scores  on  tests  of  general  mental 
abilities,  the  higher  the  average  job  performance  (Schmidt  and  Hunter, 
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1981).  Also,  while  it  is  entirely  possible  that  all  the  graduates  of  a 
particular  course  of  instruction  might  reach  an  acceptable  level  of  job 
performance,  the  average  performance  of  the  selected  group  (assuming  a 
selection  ratio  of  less  than  1,0)  will  be  higher  if  only  the  most  able  are 
selected. 

Having  said  all  this  we  must  keep  in  mind  that  it  is  another  question  as  to 
whether  the  gains  in  resultant  performance  from  top  down  selection  are 
worth  the  costs.  For  example,  is  the  aggregate  gain  in  performance  worth 
the  cost  of  testing?  If  adverse  impact  is  designated  as  a  cost  then  we  can 
also  ask  whether  the  gains  in  average  performance  are  worth  the  costs  of 
adverse  impact  that  are  incurred. 

CRITERIOK  BIAS 

Are  there  group  differences  on  measures  of  job  performance  that  are  not 
accounted  for  by  actual  differences  in  job  performance?  If  there  is 
significant  racial  or  gender  bias  in  the  criterion  then  empirical  validity 
research  will  tend  to  emphasize  predictors  that  are  also  biased.  The 
principal  conclusion  that  well  developed  tests  of  general  mental  abilities 
are  not  biased  against  blacks  rests  on  the  prior  conclusion  that  the 
criterion  measures  used  in  personnel  research  incorporate  no  serious  racial 
bias.  The  data  on  this  question  are  reviewed  by  Arvey  (1977)  and  Schmitt 
and  Lappin  (1980).  While  there  is  some  data  to  suggest  that  raters  tend  to 
give  higher  performance  ratings  to  members  of  their  own  race,  there  is  also 
evidence  suggesting  the  opposite  (i.e.,  or  reverse  discrimination  effect). 
However,  the  preponderance  of  the  evidence  suggests  no  significant 
difference. 
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RELEVANCE  FOR  PROJECT  A 


During  the  past  20  years  a  body  of  professional  and  public  policy  has  been 
developed  that  governs  the  conduct  of  personnel  decision  making  relative  to 
equal  employment  opportunity.  The  relevant  statute  is  the  Civil  Rights 
Act  of  1964.  The  principal  administrative  policy  guidelines  are  the 
Uniform  Guidelines  for  Employment  Selection.  Professional  policy  is 
contained  in  the  Joint  Standards  for  the  Use  of  Psychological  Tests, 
published  jointly  by  the  American  Psychological  Association,  the  National 
Council  of  Measurement  in  Education,  and  the  American  Education  Research 
Association,  and  the  Principles  for  the  Use  of  Tests  in  Personnel  Selection 
(Rev.  Ed.),  published  by  the  Division  of  Industrial  and  Organizational 
Psychology  of  the  American  Psychological  Association. 

Project  A,  as  it  was  designed  and  as  it  is  being  carried  out,  will  meet  all 
the  standards  imposed  by  the  federal  government  and  by  the  policy  documents 
of  the  relevant  professions.  In  fact,  it  will  go  beyond  these  standards  in 
an  attempt  to  maximize  both  the  effectiveness  of  the  Army's  personnel 
system  and  the  employment  opportunities  of  all  individuals.  Because  of  its 
comprehensive  design,  large  sample  sizes,  and  multiple  methods  of  measure¬ 
ment,  Project  A  is  in  a  better  position  to  deal  with  EEO  issues  than  any 
previous  selection/classification  research  study.  It  is  incumbent  on 
everyone  to  help  the  project  realize  its  fullest  potential  in  this  regard. 
What  follows  is  simply  a  list  of  the  many  different  elements  in  Project  A  ^ 
that  have  relevance  for  EEO  issues. 

A,  Mean  predictor  and  criterion  differences  for  blacks/whites  and 

males/females  will  be  available  for  both  new  recruits  and  experienced 
job  incumbents  for  several  jobs  and  for  a  broad  range  of  predictor 
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variables  and  criterion  variables.  One  of  the  strengths  of  Project  A 
in  this  regard  is  that  an  exhaustive  search  was  made  of  all  potential 
predictors  and  all  potential  criteria  and  a  large  sample  of  measures 
from  each  domain  was  selected  without  regard  to  the  specific  set  of 
jobs  to  be  studied.  Consequently,  we  have  an  excellent  opportunity  to 
determine  how  the  mean  difference  between  groups  varies  across 
virtually  the  complete  range  of  predictors  and  criteria  that  are 
available  to  personnel  specialists.  If  there  are  types  of  variables 
for  which  adverse  impact  is  significantly  reduced  we  should  be  able  to 
identify  them.  Maximizing  this  opportunity  means  that  we  should  retain 
the  widest  possible  variety  of  measures  through  as  many  data  collec¬ 
tions  as  possible. 

B.  In  addition  to  looking  at  mean  difference  we  can  examine  differential 
prediction  across  race  and  gender  groups  for  a  wide  range  of  predictor/ 
criterion  combinations.  If  there  are  particular  combinations  which 
yield  reasonable  validity  but  low  adverse  impact  we  should  be  able  to 
find  them.  Project  A  should  go  a  long  way  toward  a  definitive  answer 
regarding  the  extent  to  which  it  is  potentially  possible  in  employment 
settings  to  minimize  adverse  impact  and  maximize  validity  at  the  same 
time. 

C.  Within  Project  A  the  form  of  the  prediction  function  can  also  be 
examined  for  the  widest  variety  of  predictor/criterion  combinations 
ever  studied  in  a  single  project.  For  example,  we  can  ask  whether 
there  is  in  fact  any  particular  class  of  criteria  for  which  regression 
is  nonlinear. 
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D.  It  seems  generally  agreed  that  the  most  powerful  method  for  examining 
Item  bias  across  racial  or  gender  groups  is  via  item  response  theory 
techniques.  For  at  least  some  MOS  we  should  be  able  to  use  these 
techniques  to  calibrate  items  in  both  groups. 

E.  Project  A  is  attempting  to  develop  a  utility  metric  by  which  the 
payoffs  from  classification  under  various  conditions  can  be  compared. 

If  a*reasonable  metric  can  be  developed  then  it  will  be  possible  to 
portray  very  systematically  the  tradeoffs  between  more  or  less  adverse 
impact  and  the  aggregate  gain  or  loss  in  system  utility.  The 
comparative  effects  of  random  selection,  random  selection  above  a 
certain  cutting  score,  etc.,  can  also  be  simulated  to  portray  the 
effects  of  a  wide  variety  of  value  judgments  and  selection  policy  on 
the  aggregate  utility  function. 

The  above  elements  will  come  together  in  essentially  the  following 
fashion.  The  first  step  is  to  develop  criterion  composites  that  maximize 
content  and  construct  validity  as  comprehensive  measures  of  enlisted  job 
performance.  That  is,  we  want  the  best  possible  assessment  of  job 
performance  without  regard  to  how  well  each  component  might  be  predicted. 

At  the  same  time,  we  must  use  all  available  evidence  to  insure  that  the 
criterion  components  do  not  incorporate  racial  or  gender  bias.  The 
principal  means  for  doing  this  are  to  show:  1)  that  any  group  differences 
that  do  exist  are  consistent  across  different  methods  and/or  different 
groups  of  raters  and  2)  that  a  panel  of  experts  does  not  judge  the  content 
of  the  criterion  measures  to  be  biased.  The  next  step  is  to  select 
predictor  composites  that  maximize  classification  validity  and  minimize 
adverse  impact.  If  adverse  impact  still  exists  but  there  is  valid  and 
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unbiased  prediction  of  the  performance  criteria  then  a  value  judgment  must 
be  made  as  to  the  tradeoff  between  reducing  adverse  impact  vs.  reducing  the 
aggregate  performance  of  those  selected  and  classified.  If  an  appropriate 
performance  utility  metric  can  be  developed  it  will  greatly  facilitate  the 
meaningful  description  of  the  decision  options. 

A  final  warning  Is  In  order.  Interpretations  of  group  differences  Is 
fraught  with  psychometric  pitfalls  (cf  Movicic,  1984).  Biased  sampling  from 
the  reference  population  and  regression  effects  due  to  unreliability  can 
complicate  the  tasic  of  drawing  conclusions  from  data.  We  must  all  proceed 
with  caution. 

SUMMARY 

This  memo  Is  meant  to  be  only  a  bare  outline  of  EEO  Issues  as  they  pertain 
to  Project  A.  If  you  haven't  done  so  already,  every  one  should  become 
familiar  with  the  following  basic  sources: 

*  The  2-volume  report  of  the  National  Academy  of  Sciences. 

*  The  book  on  employment  fairness  by  Richard  Arvey. 

*  The  October  1981  Issue  of  the  American  Psychologist,  which  was 
a  special  issue  devoted  to  testing  and  discrimination  issues. 
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Synthetic  Validation  Project:  Pilot  Test  Results 
Project  Overview  and  Pilot  Test  Objectives 

The  two  major  objectives  of  the  Army  Synthetic  Validation  Project  are 
to  identify  and  evaluate  procedures  for: 

•  identifying  an  optimal  composite  of  selection  measures  for  any 
Army  enlisted  MOS  and  estimating  the  validity  of  this  composite 
for  predicting  job  performance 

t  setting  a  minimum  qualifying  score  so  as  to  assure  a  reasonable 
probability  of  successful  job  performance,  as  well  as  other 
appropriate  cutting  scores  for  other  critical  selection  decisions 
(e.g.,  for  selecting  recruits  with  potential  for  outstanding 
performance) . 

Synthetic  validation  approaches  typically  begin  with  the 
identification  of  a  set  of  job  components  that  can  be  used  to  describe  the 
population  of  jobs  being  studied.  A  prediction  equation  is  derived  for 
linking  available  selection  tests  to  each  component.  Subject  matter 
experts  (SMEs)  are  asked  to  judge  the  importance  of  each  component  to 
overall  job  performance.  Finally,  the  prediction  equations  for  the  various 
components  are  weighted  according  to  the  judgment  weights  and  summed  to 
obtain  an  equation  for  predicting  overall  performance  for  the  job. 

As  a  first  step  in  developing  procedures  for  obtaining  overall 
performance  prediction  equations,  we  reviewed  several  alternative 
approaches  to  constructing  a  taxonomy  of  job  components.  Following  the 
review,  we  proceeded  with  initial  development  of  three  job  component 
models.  The  job  descriptors  used  in  the  first  model  were  task  categories, 
those  used  in  the  second  model  were  job  activities  or  behaviors,  and  those 
used  in  the  third  model  were  individual  attributes  (e.g.,  abilities, 
traits,  interests)  required  for  successful  job  performance. 

To  date,  we  have  completed  a  Pre-Test  and  a  Pilot  Test  of  the  three 
models.  Results  of  the  Pre-Test  were  disseminated  at  the  Scientific 
Advisory  Committee  meeting  on  November  20,  1987.  Results  of  the  Pilot  Test 
are  summarized  in  this  report. 

/non  9°®^  0^  the  Pilot  Test  was  to  assess  the  extent  to  which  Army  SMEs 
(NCOS  and  officers)  can  reliably  identify  the  critical  task  categories, 
activities,  and  attribute  requirements  of  Army  jobs.  Other  issues  included 
identifying  possible  subgroup  differences  in  criticality  ratings  and 
identifying  ways  to  fine-tune  data  collection  procedures  for  each  of  the 
three  models. 


Method 


Participants 

Data  were  gathered  from  a  total  of  134  SMEs  in  three  MOS  -- 
Infantryman  (IIB),  Vehicle/Generator  Mechanic  (63B),  and  Administrative 
Specialist  (71L). 


Two  populations  of  judges  participated  in  the  job  description 
workshops.  The  first  was  NCOs,  officers,  and  civilians  assigned  to  the 
Directorate  of  Training  and  Doctrine  (DOTD)  who  help  define  doctrine  and 
prepare  training  plans  for  each  MOS.  The  second  consisted  of  FORSCOM  NCOs 
and  officers  who  were  supervising  soldiers  in  these  three  MOS. 

The  DOTD  data  collection  sites  included  Fort  Benning  (IIB),  Aberdeen 
Proving  Ground  (63B),  and  Fort  Benjamin  Harrison  (711).  The  workshops  were 
conducted  separately  for  NCOs  and  for  officers. 

Two  FORSCOM  installations  were  visited  --  Fort  Stewart  and  Fort  Ord 
These  two  installations  were  selected  to  ensure  that  our  workshop 
participants  included  adequate  samples  of  both  mechanized  infantrymen  (Fort 
Stewart)  and  non-mechanized  infantrymen  (Fort  Ord). 

Tables  la  and  lb  show  the  number  of  participants  by  rank  and  by 
command  for  each  MOS.  Additional  demographic  information  can  be  found  in 
Appendix  A. 


Procedure  and  Instruments 


^  Each  workshop  began  with  the  administrator  giving  an  overview  of 
project  goals  and  a  description  of  the  specific  objectives  for  the 
workshop.  After  the  overview,  participants  completed  judgment  exercises 
that  were  organized  into  four  sessions. 


Session  1:  Task  Category  and  Job  Activity  Ratings  and  Task  Matching. 
For  each  job  component  in  the  Task  Category  and  Job  Activity  instruments 
(see  Appendices  B  and  C,  respectively),  participants  provided  importance, 
frequency,  and  difficulty  ratings.  They  also  matched  each  job  component  to 
up  to  30  Project  A  job  tasks.  Following  each  matching  exercise, 

completed  a  brief  questionnaire  which  asked  their  opinions 
about  the  clarity  and  completeness  of  instructions  and  items  of  each 
component  instrument.  The  order  of  presenting  the  two  instruments  was 
balanced  across  rank,  command,  site,  and  MOS. 


Session  2:  Attribute  Importance  and  Validity  Judgments  and  Attribute 
Ranking.  Participants  were  presented  with  definitions  of  31  attributes 
(cognitive  abilities,  perceptual -psychomotor  abilities,  temperaments, 
interests)  and  the  five  Project  A  performance  factors  (see  Appendix  D). 

They  were  asked  to  rate  the  importance  or  validity  of  each  attribute  for 
each  performance  factor.  Participants  also  were  asked  to  rank  order  the  31 
attributes  according  to  their  importance  or  validity  for  overall 
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performance.  After  ranking  the  31  attributes,  participants  completed  a 
brief  questionnaire  which  asked  their  opinions  about  the  clarity  and 
completeness  of  instructions  and  definitions  as  they  were  used  in  the 
attribute  instrument. 

Session  3:  Attribute  Importance  Judgments  for  Job  Components.  From 
Session  1,  the  administrator  selected  up  to  5  task  categories  and  up  to  5 
job  activities  with  the  highest  mean  importance  ratings.  Participants  were 
asked  to  rate  the  importance  or  validity  of  each  of  the  31  attributes  for 
each  of  these  10  components. 

Session  4:  Group  Consensus.  The  administrator  also  identified  the 
five  task  categories  and  five  job  activities  with  the  highest  standard 
deviations.  Workshop  participants  were  asked  to  discuss  their  ratings  of 
these  job  components.  After  this  discussion,  participants  rerated  the 
importance  of  these  components. 

Participants  at  Aberdeen  Proving  Ground  completed  sessions  1,  2,  and 
4,  and  participants  at  Ft.  Banning  completed  sessions  1  and  2. 

Participants  at  others  sites  completed  all  four  sessions,  1  and  2  in  the 
morning  and  3  and  4  in  the  afternoon. 

Results:  Task  Category  and  Job  Activity 
Descriptive  Statistics 

One  of  the  goals  of  the  Pilot  Testing  was  to  assess  whether  judges  can 
determine  the  relative  importance,  frequency,  and  difficulty  of  each  job 
component  for  their  MOS.  As  the  three  MOS  have  rather  different  job 
duties,  it  was  critical  that  the  judgments  obtained  for  the  task  categories 
and  job  activities  reflect  the  differences  among  the  MOS. 

For  each  task  category  and  job  activity,  participants  gave  four 
judgments:  (a)  whether  the  component  was  part  of  the  job,  (b)  importance, 
(c)  frequency,  and  (d)  difficulty.  When  a  participant  rated  the  component 
"not  part  of  the  job,"  a  value  of  zero  was  entered  for  the  participant’s 
importance  and  frequency  ratings. 

Task  Categories.  Tables  2a-2c  show,  for  each  MOS,  means  and  standard 
deviations  for  the  importance,  frequency,  and  difficulty  ratings  for  the  97 
task  categories.  Tables  3a-3c,  show  for  each  MOS,  the  task  categories 
ordered  by  the  mean  importance  ratings.  The  results  in  Tables  3a-3c  show 
that  tasks  that  are  expected  to  be  important  for  the  MOS  received  the 
highest  importance  ratings.  For  IIB,  items  on  combat,  weapons,  explosives, 
first  aid,  navigation,  and  communications  received  highest  importance 
ratings.  For  63B,  items  on  using  manuals,  repairing,  troubleshooting, 
vehicle  operation,  and  general  soldiering  items  were  rated  most  important. 
For  71L,  typing,  document  preparation  and  handling,  and  general  soldiering 
items  received  the  highest  ratings.  In  all  three  MOS,  a  common  set  of 
general  soldiering  task  categories  such  as  navigate,  give  first  aid,  use 
maps,  and  fire  individual  weapons  were  among  the  twenty  most  important 
components. 
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Zero  order  correlations  of  the  mean  importance  rating  profiles  between 
pairs  of  MOS  were  .66  for  IIB  and  63B,  .48  for  IIB  and  71L,  and  .40  for  63B 
and  71L.  These  moderate  correlations  provide  more  support  that  our  judges’ 
ratings  can  be  used  to  identify  differences  in  critical  tasks  between  the 
three  MOS.  The  profile  of  importance  weights  was  more  similar  between  IIB 
and  63B  than  either  MOS  with  71L. 


Although  standard  deviations  for  the  mean  importance  ratings  ranged 
from  .13  to  2.24,  the  vast  majority  of  them  were  less  2.0.  Larger  standard 
deviations  were  generally  associated  with  components  that  received  lower 
importance  ratings. 


jpb  Activities.  Tables  4a-4c  show,  for  each  MOS,  means  and  standard 
deviations  for  the  importance,  frequency,  and  difficulty  ratings  for  the  53 
job  activities.  Tables  Sa-Sc  show,  for  each  MOS,  the  activities  ordered  by 
the  mean  importance  ratings.  Again  activities  expected  to  be  most 
important  for  each  MOS  were  indeed  rated  most  important.  For  IIB,  items 
dealing  with  combat,  weapons,  physical  activities,  and  following  directions 
received  the  highest  importance  ratings.  For  63B,  items  on  operating  hand 
tools  and  equipment,  driving,  troubleshooting,  and  following  directions 
received  the  highest  importance  ratings.  For  71L,  items  on  operating 
keyboard,  following  or  relaying  directions,  teamwork,  and  solving 
administrative  problems  received  the  highest  importance  ratings. 


Mnc  I  ^  importance  rating  profiles  between  pairs  of 

MOS  were  .61  for  IIB  and  63B,  .35  for  IIB  and  71L,  and  .28  for  63B  and  71L 
These  moderate  correlations  provide  greater  support  that  our  judges’ 
rating  can  be  used  to  identify  differences  in  critical  activities  between 
the  three  MOS.  The  profile  of  importance  weights  was  more  similar  between 
IIB  and  63B  than  either  MOS  with  71L. 


Although  standard  deviations  for  the  mean  importance  ratings  ranged 
from  .43  to  2.20,  most  were  less  than  2.0.  Again,  larger  standard  ^ 
deviations  were  generally  associated  with  activities  that  received  lower 
importance  ratings. 

LQrrelations  among  judgments.  Table  6a  shows  the  mean  within-SME 
correlations  between  the  importance,  frequency,  and  difficulty  ratings  for 
the  task  categories  that  were  rated  "part  of  the  job."  In  other  words, 
these  are  correlations  between  non-missing  ratings  only.  The  correlations 
are  reported  separately  for  each  MOS.  The  mean  within-SME  correlations 
between  importance  and  frequency  ratings  were  r=.48  for  IIB,  r=.57  for  63B 
and  r=.63  for  71L.  The  mean  within-SME  correlations  between  Difficulty  and 
Importance  or  Frequency  ratings  ranged  from  r=-.09  to  r=.25. 

Table  6b  shows  the  mean  within-SME  correlation  between  the  importance, 
frequency,  and  difficulty  ratings  for  the  job  activities  that  were  rated 
part  of  the  job.  The  correlations  are  reported  separately  for  each  MOS 
The  mean  within-SME  correlations  between  importance  and  frequency  ratings 
were  r=.56  for  IIB,  r=.70  for  63B,  and  r=.70  for  71L.  The  mean  within-SME 
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correlations  between  difficulty  and  importance  or  frequency  ratings  ranged 
from  r=-.02  to  r=.21. 

These  results  suggest  that,  on  both  the  task  and  activity  instruments, 
there  was  some  redundancy  in  importance  and  frequency  information,  but  the 
difficulty  ratings  yielded  rather  different  information.  The  difficulty 
ratings  could  serve  as  a  tie-breaker  for  ordering  components  that  are  equal 
in  importance  or  frequency. 


Variance  Component  and  Reliability  Estimates 


Variance  component  estimates  provide  a  concise  way  of  summarizing  the 
differences  among  job  descriptor  means  and  describing  the  relative 
contribution  of  various  conditions  to  variance  in  the  ratings.  Independent 
variables  and  the  number  of  levels  within  each  variable  are  as  follows: 


Condition 

Component 

MOS 

Rank 

Command 

Rater 


Number  of  Levels 

97  for  Task  Category;  53  for  Job  Activity 
3  (IIB,  63B,  71L) 

3  (NCO,  Officer,  Civilian) 

2  (DOTD,  FORSCOM) 

134 


As  rater  effects  were  nested  within  MOS,  rank,  and  command,  we  ran  two 
separate  analysis  of  variance  models.  The  first  model  included  only  rater 
and  component.  Estimates  of  the  variance  among  all  raters  were  obtained 
from  this  model.  The  second  model  included  component,  MOS,  rank,  and 
command,  but  not  rater.  Variance  attributable  to  terms  with  MOS,  rank, 
and/or  command  were  subtracted  from  the  overall  rater  variance  estimates 
from  the  first  model.  The  differences  provided  estimates  of  the  variance 
due  to  rater  nested  within  MOS,  rank,  and  command. 

Tables  7  to  12  provide  variance  component  and  reliability  estimates 
for  the  task  category  and  job  activity  importance  ratings.  Each  table  is 
based  on  a  different  variance  component  model: 

(1)  Component,  MOS,  Rank,  Command,  and  Rater  (Table  7) 

(2)  Component,  MOS,  Command,  and  Rater  for  each  Rank  (Tables  8a-8b) 

(3)  Component,  MOS,  Rank,  and  Rater  for  each  Command  (Tables  9a-9b) 

(4)  Component,  Rank,  Command,  and  Rater  for  each  MOS  (Tables  10a- 10b) 

(5)  Component,  Command,  and  Rater  for  each  MOS  and  Rank  (Tables  lla-lld) 

(6)  Component,  Rank,  and  Rater  for  each  MOS  and  Command  (Tables  12a-12d) 

From  results  in  Tables  7  to  12,  we  underscore  the  following 
observations: 


the  large  variance  component  for  both  task  category  and  job 
activity  showed  that,  within  MOS,  judges  reliably  discriminated 
the  important  job  components  from  the  less  important  ones  (see 
Tables  10a  and  10b) 
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•  the  large  variance  component  for  the  Component  x  MOS  interaction 
suggested  that  weights  were  applied  differentially  for  the  job 
components  across  MOS  (see  Table  7) 

Another  important  goal  of  the  Pilot  Testing  was  to  assess  whether  SMEs 
can  accurately  and  reliably  apply  the  instruments  in  describing  jobs. 

Table  13a  is  a  summary  table  of  the  average  single-SME  reliability 
estimates  for  both  the  task  category  and  job  activity  instruments  found  in 
Tables  7  through  9*  The  table  also  shows  the  number  of  judges  required  to 
reach  a  target  reliability  of  .90.  The  overall  single-SME  reliabilities  of 
.58  for  the  task  instrument  and  .45  for  the  activity  instrument  are 
respectable  reliabilities. 

Subgroup  differences 


Another  issue  in  this  research  was  the  selection  of  SMEs  for 
describing  jobs.  We  were  interested  in  determining  whether  judges  of 
various  ranks  or  from  different  assignments  evaluated  the  jobs  similarly. 
The  research  design  included  about  equal  numbers  of  NCOs  and  officers  and 
about  equal  numbers  of  FORSCOM  and  DOTD  judges. 


The  reliabilities  in  Table  13b  and  13c  show  that  officers  were 
somewhat  more  reliable  than  NCOs  for  all  three  MOS  and  on  both  instruments. 
Overall  reliabilities  for  DOTD  and  FORSCOM  were  generally  comparable.  DOTD 
judges  were  somewhat  more  reliable  than  FORSCOM  judges  for  IIB  and  63B. 

For  71L,  FORSCOM  judges  were  somewhat  more  reliable  than  DOTD  judges. 


Although  there  were  small  differences  in  reliabilities  for  different 
ranks,  the  rank  ordering  of  task  categories  and  activities  was  very 
consistent  across  ranks  and  commands.  The  correlation  of  mean  task 
category  importance  rating  profiles  was  .96  for  IIB  NCOs  and  officers,  .95 
for  63B  NCOS  and  officers,  and  .95  for  71 L  NCOs  and  officers.  Comparable 
correlations  for  the  job  activities  were  .92,  .82,  and  .94  for  IIB,  63B, 
7lL,  respectively.  For  the  task  categories,  profile  correlations  for 
DOTD  and  FORSCOM  were  .95  for  IIB,  .91  for  63B,  and  .93  for  71L. 

Comparable  correlations  for  the  activity  ratings  were  .95,  .79,  and  .95, 
respectively. 


In  sum,  reliabilities  for  both  DOTD  and  FORSCOM  were  comparable, 

slightly  more  reliable  than  NCOs  in  their  ratings. 
Also,  NCOS  and  officers  and  DOTD  and  FORSCOM  judges  provided  highly  similar 


Pilot  Test  V.  Pre-Test  Rel iahilit.ip<; 

X  un  Corresponding  Pre-Test  reliabilities  have  also  been  included  in  the 
Table  13b  and  13c.  For  IIB  Pre-Test  reliabilities  were  higher  than  Pilot 
Test  reliabilities.  For  71L,  Pilot  Test  reliabilities  were  higher  than 
Pre-Test  values.  For  63B,  Pilot  Test  reliabilities  were  higher  for  the 
task  instrument,  but  there  were  no  reliability  differences  for  the  activity 
instrument.  ^ 
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Task  Matching 


The  purpose  of  the  Task  Matching  Exercise  was  twofold:  (1)  to  provide 
an  indication  of  the  exhaustiveness  of  each  job  component  model  for 
describing  jobs;  and  (2)  to  provide  necessary  linkages  (via  expert 
judgment)  to  enable  the  comparison  between  synthetic  (inferred)  validities 
and  empirical  validities  obtained  from  Project  A. 

Results  of  the  task  matching  exercise  are  reported  in  Tables  14a-14c. 
(Detailed  results  are  reported  in  Appendix  E.)  The  first  column  of  Tables 
14a-14c  lists  up  to  30  Project  A  tasks  for  each  MOS.  For  each  task,  SMEs 
identified  up  to  three  task  categories  and  up  to  three  activities  that  best 
matched  it.  The  remaining  columns  show  the  job  component  number  and  the 
proportion  who  selected  that  component  as  one  of  their  three  task 
categories  or  three  activities.  Only  job  components  selected  by  50%  or 
more  of  the  SMEs  are  tabulated. 

We  found  that:  (1)  SMEs  could  more  readily  match  the  job  tasks  to  the 
task  categories  than  to  the  job  activities;  (2)  the  SMEs  used  more  task 
categories  than  job  activities  to  cover  the  Project  A  tasks  (e.g.,  71Ls 
used  13  task  categories  and  only  2  activities);  (3)  there  were  several 
tasks  that  were  not  covered  by  any  of  the  task  categories  or  activities; 
and  (4)  in  several  cases,  content  areas  that  were  not  covered  by  one  model 
were  covered  by  the  other. 

Tables  15a  and  15b  show  the  proportion  of  the  important  task 
categories  and  job  activities,  respectively,  that  were  matched  to  the 
Project  A  tasks.  It  is  clear  that  the  number  of  important  task  categories 
that  were  matched  to  tasks  were  far  greater  than  the  number  of  important 
job  activities  that  were  matched  to  tasks.  These  results  suggest  that  we 
may  be  unable  to  conduct  comparisons  between  empirical  validities  and 
synthetic  validities  provided  derived  for  activity-based  job  components. 

The  task  matching  exercise  data  suggest  that: 

t  ambiguities  still  exist  in  several  of  the  job  component  definitions 
(particulary  in  the  use  and  meaning  of  the  examples  provided) 

•  the  SMEs  clearly  were  able  to  match  task  to  task  categories  more 
easily  with  task  categories  than  with  job  activities 

•  the  definitions  of  some  of  the  job  activities  may  be  too  broad  to 
link  with  specific  tasks  (e.g.,  the  activity  "Follow  written 
directions"  was  mapped  to  both  "Type  a  military  letter"  and  "Put  on 
fi el  d/pressure  dressing"  for  71L) 

•  neither  model  did  a  good  job  describing  many  of  the  IIB  Project  A 
combat  tasks 
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Evaluation  of  Instruments 


After  the  task  matching  exercise,  a  brief  questionnaire  was  presented 
to  judges  for  them  to  evaluate  the  task  and  the  activity  instruments. 

Copies  of  the  evaluation  questionnaires  are  included  in  Appendices  B  and  C. 
Both  ratings  and  open-ended  comments  were  gathered  from  judges.  One  of  the 
open-ended  question  asked  judges  to  suggest  components  which  should  be 
deleted  or  added.  Those  who  responded  to  the  question  had  suggestions 
which  were  either  too  specific  or  too  broad.  There  was  also  an 
insufficient  number  of  comments  to  inform  any  decision.  Other  written-in 
general  evaluations  by  judges  were  considered  in  revising  the  instruments. 
Means  and  distribution  of  judges’  rating  evaluation  of  the  task  instrument 
and  the  activity  instrument  are  summarized  in  Tables  16  and  17, 
respectively.  Here  are  some  key  observations: 

•  SMEs  reported  that  the  instructions  on  both  instruments  were  clear 

•  although  SMEs  reported  that  the  terms  and  definitions  on  both 
instruments  were  clear  and  easy  to  understand,  SMEs  found  them 
somewhat  clearer  and  easier  to  understand  on  the  task  instrument 
than  on  the  activity  instrument 

•  SMEs  reported  that  the  ratings  were  neither  easy  nor  difficult  to 
make 

•  SMEs  were  generally  confident  about  the  accuracy  of  their  ratings 

t  except  for  63B  on  the  task  instrument,  both  instruments  provided, 
on  the  average,  about  75%  coverage  of  the  jobs;  for  63B,  the  task 
instrument  provided  91%  coverage 

t  subgroup  comparisons  between  MOS,  rank,  and  command  did  not  reveal 
any  large  systematic  differences 

Conclusions:  Task  Category  and  Job  Activity 

The  Pilot  Test  results  confirmed  that  Army  SMEs  can  reliably  identify 
the  critical  task  categories  and  activities  of  their  jobs.  Subgroup 
analyses  between  NCOs  and  officers  and  between  DOTD  and  FORSCOM  showed 
essentially  no  differences  in  subgroup  ratings.  There  was  no  conclusive 
evidence  that  any  one  job  component  model  is  vastly  superior  to  the  others 
for  describing  jobs. 

The  results  suggested  ways  the  data  collection  could  be  improved. 

Based  on  the  Pilot  Test  experience,  we  have  made  several  changes  for  Phase 
I  data  collection  procedures.  These  changes  include: 

§  elimination  of  difficulty  judgments  for  the  task  categories  and 
activities 
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t  separate  importance  ratings  for  Core  Technical  Proficiency,  General 
Soldiering  Proficiency,  and  overall  job  performance  for  each  task 
category  and  activity 

•  elimination  of  the  task  matching  exercise  for  task  categories  and 
activities  instruments 

•  rewording  task  category  and  activity  items  with  importance  rating 
standard  deviations  of  2.0  or  higher 

•  improved  instructions,  including  better  examples,  for  all  three 
protocols. 

•  inclusion  of  a  rating  scale  on  the  answer  sheet 

Results:  Attributes 

We  turn  now  to  a  discussion  of  the  results  obtained  from  analyses  of 
the  Attribute  ratings  and  rankings.  We  first  discuss  analyses  of  the 
responses  the  SMEs  made  on  the  evaluation  questionnaire  and  then  turn  to 
the  analyses  of  the  ratings  and  rankings  themselves. 

Analyses  of  Evaluation  Questionnaire 

Description  of  Evaluation  Questionnaire.  The  Attribute  Model 
Evaluation  Sheet  had  four  sections:  (1)  performance  definition,  (2) 
attribute  ranking,  (3)  attributes,  (4)  general  evaluation  (a  complete  copy 
is  included  in  Appendix  D).  The  performance  definition  and  attribute 
ranking  sections  included  ratings  designed  to  assess  the  clarity  of  the 
performance  definitions  and  the  ranking  task.  The  performance  definition 
and  attributes  sections  included  open-ended  questions  designed  to  elicit 
comments  concerning  additions  or  deletions  to  the  survey  materials.  The 
attributes  section  also  contained  an  item  asking  participants  to  check  any 
attributes  that  were  difficult  to  judge  or  rank.  In  the  general  evaluation 
section,  participants  were  asked  to  compare  the  judgment  and  ranking  tasks 
on  two  dimensions:  ease  of  use  and  amount  of  information  provided.  A 
third  item  in  the  general  evaluation  section  asked  participants  which  task 
they  would  choose  to  guide  selection  and  placement  of  soldiers  in  MOS. 

Evaluations  were  completed  by  127  SMEs.  The  breakdown  of  participants 
by  method,  rater  type,  MOS,  and  command  appears  in  Table  18. 

^  Performance  Definition  and  Ranking  Task  Ratings.  SMEs  indicated  that 
the  performance  definition  contained  the  right  amount  of  detail  and 
information  (4.55  on  a  7-point  scale,  with  4  representing  "the  right 
amount").  No  significant  differences  were  found  when  participants  were 
subgrouped  by  method,  rater  type,  MOS,  or  command.  Table  19  presents  the 
means  and  standard  deviations  by  subgroup. 

Overall,  instructions  for  the  ranking  task  were  rated  moderately  clear 
(5.61  on  a  7-point  scale).  Significant  differences  were  found  when 
participants  were  subgrouped  by  method  and  by  rater  type.  SMEs  who  took 
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the  Importance  Task  rated  the  ranking  instructions  as  more  clear  than  did 
participants  who  took  the  Validity  Task  (F  =  18.47,  p  <  .001).  Civilians 
rated  the  ranking  task  instructions  as  less  clear  than  did  NCOs  or 
Officers,  and  NCOs  rated  the  instructions  as  less  clear  than  did  Officers 
(F  *  3.87,  p  =  .023).  Means  and  standard  deviations  by  subgroup  are 
presented  in  Table  20. 

The  ranking  task  was  rated  neither  easy  nor  difficult  (4.27  on  a 
7-point  scale).  No  significant  differences  were  found  for  subgroups. 

Table  21  presents  the  means  and  standard  deviations  by  subgroup. 

Overall,  participants  were  moderately  confident  of  the  accuracy  of 
their  rankings  (4.92  on  a  7-point  scale).  A  significant  difference  in 
confidence  ratings  was  found  when  participants  were  subgrouped  by  method. 
Those  who  took  the  Importance  Task  were  more  confident  of  their  rankings 
than  were  those  who  took  the  Validity  Task  (F  =  5.75,  p  =  .018).  Table  22 
presents  the  means  and  standard  deviations  by  subgroup. 

No  significant  differences  between  MOS  or  command  subgroups  were  found 
for  the  performance  definition  or  ranking  task  ratings. 

Correlations  of  the  four  evaluation  ratings  are  presented  in  Table  23. 
Ratings  of  the  amount  of  detail  in  the  performance  definition  are  unrelated 
to  the  three  ranking  task  ratings.  In  the  total  sample,  the  three  ranking 
task  ratings  are  significantly  correlated  (p  <  .001).  When  the 
correlations  are  examined  separately  by  method,  the  correlation  between  the 
ranking  task  clarity  rating  and  the  ranking  task  difficulty  rating  is  not 
significant  in  the  Importance  Task  subsample,  but  is  significant  (p  <  .001) 
in  the  Validity  Task  subsample. 

Difficult  Attributes.  When  asked  to  indicate  at  least  two  attributes 
that  were  difficult  to  judge  or  rank,  13  participants  did  not  respond,  2 
participants  checked  only  one  attribute,  51  participants  checked  two 
attributes,  and  61  participants  checked  more  than  two  attributes.  The  five 
attributes  most  frequently  marked  were  Spatial  Ability  (N  =  35),  Closure  (N 
-  31),  Interest  in  Artistic  Activities  (N  -  25),  Interest  in  Protective 
Services  (N  =  24),  and  Interest  in  Science  (N  =  23).  Frequencies  for  each 
of  the  31  attributes  are  presented  in  Table  24. 

Comments.  Tables  25  through  28  present  the  performance  definition  and 
attribute  cotranents.  Most  participants  declined  to  respond  when  provided 
with  opportunities  to  comment  on  the  performance  definition  and  the 
attributes.  Several  participants  made  comments  pertaining  to  the 
attributes  when  asked  about  the  performance  definition,  which  indicates  the 
instructions  for  the  Evaluation  Sheet  should  be  clarified  if  it  is  to  be 
used  in  the  future.  Six  participants  suggested  the  performance  definition 
could  be  shortened.  Twenty-one  specific  attributes  were  suggested  for 
addition  to  the  survey.  Interest  in  Artistic  Activities  was  most 
frequently  nominated  for  deletion  (N  =  16). 

General  Evaluation.  Table  29  contains  frequencies  and  percentages  for 
the  general  evaluation  questions.  A  majority  of  participants  indicated 
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that  the  judgment  task  was  easier  than  the  ranking  task  (50%)  and  provides 
more  information  (63%).  When  asked  which  task  they  would  prefer  to  guide 
selection  and  placement  of  soldiers  in  MOS,  57%  indicated  a  preference  for 
the  judgment  task.  Although  "Both"  and  "Neither"  were  not  response 
options,  a  few  of  the  participants  added  them. 

.  Summary  of  Evaluation  Questionnaire  Analyses.  No  alarming  problems 
surfaced  in  these  evaluations.  This  was  the  first  time  that  we  had 
attempted  to  use  the  ranking  task,  and  were  gratified  that  the  raters  found 
the  ranking  task  doable  and  were  moderately  confident  that  their  rankings 
were  accurate. 


Nearly  the  same  attributes  were  identified  as  being  difficult  to  rate 
in  the  pilot  test  as  in  the  pre-test.  These  attribute  definitions  have 
received  considerable  attention,  and  it  is  our  opinion  that  they  can  not  be 
simplified  further  without  losing  their  content  or  construct  validity  (that 
is,  we  could  probably  write  simpler  definitions,  but  they  would  no  longer 
be  defining  the  construct  they  are  intended  to  define).  These  attributes 
(Spatial  Ability,  Closure,  and  several  Interests)  are  probably  just  the 
most  difficult  subset  to  rate  of  this  set  of  31  attributes,  and  are  not 
necessarily  in  need  of  further  revision. 


Perhaps  the  most  interesting  finding  here  is  the  preference  expressed 
for  using  the  validity  or  importance  judgments  (against  the  five 
performance  dimensions)  to  "guide  selection  or  placement  of  soldiers", 
rather  than  using  the  ranking  of  attributes  (which  appears  to  be  less 
complex  and  takes  less  time  for  the  rater)  for  that  purpose. 

..  those  interested.  Appendix  F  contains  a  summary  of  responses  to 

the  four  evaluation  questions  described  in  Tables  19-22. 


AnalYse$  of  Attribute  Ratings  and  Rankings  of  Importance  and  Validity 

analyses  of  the  judgments  made  about  the  Attributes 
by  the  SMEs.  We  first  examine  the  impact  of  various  factors  such  as  MOS, 
Type  of  rater,  and  the  Attributes  themselves  on  the  judgments  made  by  the 
SMEs.^  We  then  look  at  the  reliability  of  the  judgments.  Convergent  and 
discriminant  validity  of  the  judgments  are  examined  via  multi-trait,  multi¬ 
method  logic.  The  relationships  of  the  validity  and  importance  ratings  to 
the  validity  and  importance  rankings  are  also  examined.  Finally,  we 
examine  the  mean  values  of  the  ratings  and  rankings  in  some  detail. 


^  Lumber  of  Available  Raters.  Tables  30  and  31  show  the  number  of 
^ ®  >^ating  method  (Importance  or  Validity),  Rater  Type 
(NCO,  Officer,  or  Civilian),  and  MOS  (IIB,  63B,  or  71L).  Two  things  are 

^^®  for  the  two  tables  sum  to 

more  than  the  total  number  of  raters  available  (72  +  91  =  163  v  134)  This 

occurs  because  some  raters  completed  both  validity  and  importance  ratings. 
S®cond,  the  N  s  for  civilians  were  extremely  low  for  both  methods,  and  the 
N  for  63B  was  ptremely  low  for  the  Importance  method.  These  facts  limited 
the  analyses  that  could  be  completed,  particularly  the  Analyses  of 
Variance.  We  could  complete  the  "full-blown"  ANOVA  for  only  the  validity 
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ratings  without  civilians,  if  we  wished  to  have  a  minimum  cell  size  of  5, 
which  we  did. 


Analyses  of  Variance  of  Validity  Ratings  and  Rankings.  Tables  32-37 
show  the  results  of  ANOVAs  of  the  validity  ratings  for  each  of  the  five 
performance  areas  and  the  validity  rankings.  There  were  two 
between-subjects  factors  (MOS  and  Type)  and  one  within-subjects  factor 
(Attribute) . 

Examination  of  these  tables  shows  that  there  was  a  highly  significant 
Attribute  effect  for  all  five  performance  areas  and  for  the  rankings. 

Thus,  the  attributes  did  receive  different  ratings  and  rankings  when 
averaged  across  all  three  MOS  and  both  raters.  The  Attribute  x  MOS  effect 
was  highly  significant  for  the  Core  Technical  performance  area  (Table  32) 
and  for  the  ranking  (for  overall  job  performance,  Table  37),  and  much  less 
significant  for  the  Physical  Fitness/Military  Bearing  area  (Table  36).  It 
did  not  reach  the  .01  level  of  significance  for  the  other  three  performance 
areas.  This  finding  accords  with  our  a  priori  expectations  and  with  Project 
A  empirical  validity  results,  that  is,  there  are  validity  differences  for 
attributes  with  respect  to  the  Core  Technical  part  of  soldier’s  jobs,  but 
not  with  respect  to  the  "common"  parts  of  a  soldier’s  job--which  are 
represented  by  the  other  four  performance  areas. 

There  are  small  but  statistically  significant  Attribute  x  Type  effects 
for  all  five  performance  areas,  but  not  for  ranking.  This  finding 
indicates  that  there  may  be  sufficient  rater  type  differences  (NCO  v. 
Officer  only,  in  these  analyses)  to  warrant  including  both  types  of  raters 
in  operational  ratings.  Or  we  may  wish  to  study  the  exact  nature  of  the 
differences  in  the  ratings  given  by  NCOs  v.  Officers  and  choose  to  use  only 
one  type  in  the  operational  setting. 

Calculation  of  Inter-Rater  Agreement  Reliabilities.  We  computed 
inter-rater  agreement  reliability  coefficients  for  validity  ratings  and 
ranking  (Table  38)  and  for  importance  ratings  and  ranking  (Table  39).  The 
coefficients  were  computed  using  the  output  from  a  one-way  repeated 
measures  ANOVA  to  estimate  mean  squares  for  attributes  and  for  error;  rater 
main  effect  was  thus  included  as  part  of  the  error  term.  Note  that  we 
calculated  a  reliability  for  each  MOS  x  Performance  area  x  Rater  type  cell. 
With  one  exception  (63B  NCOs  and  Officers:  Importance,  in  Table  39)  we  only 
calculated  reliabilities  where  the  cell  N  was  at  least  10  in  order  to 
achieve  minimum  stability  for  mean  square  estimates. 

As  Tables  38  and  39  show,  the  ratings  and  rankings  are  reasonably 
reliable  for  both  importance  and  validity.  The  smallest  reliability  in 
ofr  number  of  raters,  =  15),  and  almost  half  (39 

.  .1.®*^®  greater.  There  does  not  appear  to  be  a  practically 

significant  difference  in  reliability  between  the  two  methods,  but  if  there 
IS  a  slight  edge,  it  probably  favors  the  importance  method.  However,  there 
were  generally  fewer  data  points  available  for  the  importance  method  and 
this  slight  edge  might  disappear  if  more  data  were  available. 
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There  is  not  enough  Information  in  these  tables  to  compare  Officer  and 
NCO  ratings  for  all  MOS  and  both  methods,  but  what  information  is  there 
indicates  that  Officers  generally  agree  better  than  do  NCO’s. 

Multi -Trait,  Multi -Method  Analyses.  If  one  thinks  of  MOS  as  traits 
and  the  Importance  and  Validity  ratings  as  two  different  methods,  then 
these  data  can  be  analyzed  via  the  logic  of  the  multi-trait,  multi -method 
approach.  Tables  40-46  show  the  results  of  such  analyses  for  the  five 
performance  areas  separately,  the  rankings,  and  all  five  areas  at  once 
(that  is,  using  the  full  155  item  profile  that  is  made  up  of  all  Attribute 
(31)  X  performance  area  (5)  ratings).  These  striking  results  are 
summarized  below: 


Summary  of  Multi-trait,  Multi-method  Analyses 

Means  of  the  Correlations  displayed  in  Convergent  Validity  (Bet. 
Method,  Within  MOS)  and  Discriminant  Validity  (Bet.  Method,  Between  MOS  and 
Within  Method,  Bet.  MOS)  Portions  of  Tables  40-46:  Core  Technical,  General 
Soldiering,  Effort/Leadership,  Personal  Discipline,  and  Physical 
Fitness/Military  Bearing  Performance  Areas,  Overall  Ranking,  and  All 
Performance  Areas. 


CORE 

GEN 

EFF 

DIS 

FIT 

RNK 

ALL 

Bet.  Method,  Within  MOS 

00 

00 

00 

.89 

.94 

.92 

.91 

Bet.  Method,  Between  MOS 

CM 

.83 

00 

• 

.85 

.92 

.46 

.79 

Within  Method,  Bet.  MOS 

.42 

.83 

.84 

.86 

.92 

.45 

.79 

*  Within  Importance 

.38 

.80 

.78 

.81 

.93 

.36 

.81 

*  Within  Validity 

.45 

.84 

.90 

.90 

.92 

.54 

.75 

Note  that  the  ratings  for  the  Core  Technical  Performance  Area  and  the 
Overall  Rankings  show  very  high  mean  coefficients  for  convergent  validity, 
as  do  all  the  other  areas.  Note,  further,  that  the  mean  coefficients  for 
discriminant  validity  are  forty  points  lower  for  these  two  ratings  than  for 
the  other  four  performance  areas  (.42  and  .46  versus  .83-. 92  for  the 
between  method,  between  MOS  mean  coefficients  and  .42  and  .45  v.  .83-. 92 
for  the  within  method,  between  MOS  mean  coefficients),  indicating  that 
different  profiles  of  attributes  are  obtained  for  the  three  MOS  for  Core 
iGchnical  and  for  Overall  Ranking >  but  not  for  the  other  four  performance 
areas--that  is,  discriminant  validity  exists  for  these  two  but  not  for  the 
other  four.  As  we  stated  earlier,  these  are  precisely  the  results  we  would 
expect  based  on  prior  research  conducted  by  Project  A  regarding  the 
validity  relationships  between  predictors  and  these  five  performance  areas. 

The  last  two  rows  of  the  summary  show  that  there  is  slightly  better 
discriminant  validity  for  Importance  ratings  than  for  Validity  ratings  (.36 
V.  .45  for  Core  Technical  and  .36  v.  54  for  Rankings).  Again,  we  urge 
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caution  in  overinterpreting  these  relatively  small  differences,  given  the 
size  of  the  present  samples. 

Relationship  of  Mean  Importance  and  Mean  Validity  Ratings  tn  Rankinn«; 
Tables  47-49  show  the  correlations  of  the  mean  importance  and  mean  validity 
rating  profiles  (that  is,  the  mean  rating  given  to  each  of  the  thirty-one 
attributes)  with  the  mean  importance  and  mean  validity  ranking  profiles 
(that  is,  the  mean  ranking  assigned  to  each  of  the  thirty-one  attributes) 
for  the  three  MOS.  A  separate  correlation  has  been  computed  for  each 
performance  area.  These  tables  represent  a  sort  of  rough  "policy¬ 
capturing"  of  the  rankings  of  the  attributes  for  overall  performance  in  the 
MOS.  That  is,  if  the  mean  rating  profile  for  Core  Technical  correlates 
very  highly  with  the  mean  ranking  profile  and  the  mean  rating  profile  for 
Fitness/Bearing  correlates  very  lowly  with  the  mean  ranking  profile,  then 
we  might  conclude  that  the  Core  Technical  performance  area  is  "more 
important"  for  overall  performance  than  is  Fitness/Bearing--"more 
important"  in  the  sense  that  the  attributes  rated  as  highly  and  lowly  valid 
for  Core  Technical  are  also  ranked  similarly  highly  and  lowly  valid  for 
Overall  Performance. 

Examination  of  these  three  tables  shows  that: 

•  For  IIB,  Core  Technical  and  General  Soldiering  are  nearly  equal  and 
higher  than  the  other  three- -but  the  other  three  are  still 
relatively  high. 

t  For  63B,  Core  Technical  is  much  greater  than  any  of  the  others. 
General  Soldiering  is  moderately  high,  and  the  others  are  very  low. 

•  For  71L,  Core  Technical  is  extremely  high.  General  Soldiering, 
Effort/Leadership,  and  Personal  Discipline  are  about  30  points 
less,  but  still  fairly  high,  and  Fitness/Bearing  is  zero. 

•  In  all  three  tables,  the  validity  and  importance  ratings  show  the 
same  pattern  of  relationship  with  the  rankings,  but  the  validity 
ratings  consistently  correlate  lower  with  the  rankings  than  do  the 
importance  ratings.  This  could  indicate  that  validity  ratings  are 
less  susceptible  to  an  overall  "halo"  effect  than  are  the 
importance  ratings. 

Mean  Importance  and  Mean  Validity  Rating  and  Ranking  Profilps.  Mean 
importance  and  validity  ratings  for  the  31  attributes  were  computed  by  MOS 
for  each  of  the  five  job  areas  (Tables  50-54  and  56-60).  We  then  compared 
'^®^^’^9_Pf'ofil6s  across  MOS,  but  within  job  performance  area. 
Attributes  for  which  the  maximum  difference  in  means  between  any  pair  of 
MOS  was  equal  to  or  greater  than  2.5  were  noted.  (Although  this  size  of 
mean  difference  was  somewhat  arbitrarily  chosen,  the  standard  error  of 
difference  for  almost  all  mean  comparisons  for  these  data  is  less  than  1.2. 
Therefore  the  value  of  2.5  will  almost  always  represent  a  difference 
greater  than  two  standard  errors  of  difference.)  It  was  expected  that  the 
mean  ratings  would  differ  by  MOS  for  the  Core  Technical  Proficiency  job 
area  but  not  for  other  job  areas.  This  reasoning  was  based  on  the  notion, 


14 


previously  mentioned,  that  General  Soldiering,  Effort  and  Leadership, 
Personal  Discipline,  and  Physical  Fitness/Military  Bearing  are  common 
across  MOS  and  should  consequently  be  related  in  the  same  way  to  the 
attributes  for  all  MOS,  while  Core  Technical  Proficiency  is  specific  to  the 
MOS  and,  within  each  MOS,  would  be  related  uniquely  to  the  31  attributes. 

The  mean  importance  ratings  of  the  31  attributes  did  not  correspond 
closely  to  the  expected  pattern.  Core  Technical  Proficiency  did  have  12 
attributes  showing  a  maximum  difference  at  or  above  the  2.5  cut-off: 
spatial  ability,  closure,  mechanical  comprehension,  precision,  movement 
judgment,  physical  strength,  physical  endurance,  balance  and  flexibility, 
involvement  in  athletics,  interest  in  using  tools  and  machines,  interest  in 
rugged  activities,  and  interest  in  protective  services.  However,  the  other 
four  areas  each  had  at  least  two  attributes  (rather  than  none)  that  also 
met  this  criteria.  For  General  Soldiering,  2  attributes  met  the  maximum 
difference  cut:  spatial  ability  and  involvement  in  athletics.  For  Effort 
and  Leadership,  8  attributes  met  the  maximum  difference  cut:  number 
ability,  eye-limb  coordination,  precision,  movement  judgment,  physical 
strength,  balance  and  flexibility,  interest  in  rugged  activities,  and 
interest  in  protective  services.  For  Personal  Discipline,  8  attributes  met 
the  maximum  difference  cut:  number  ability,  precision,  movement  judgment, 
physical  strength,  physical  endurance,  balance  and  flexibility,  involvement 
in  athletics,  and  interest  in  rugged  activities.  For  Physical 
Fitness/Military  Bearing,  3  attributes  met  the  maximum  difference  cut: 
dominance/confidence,  interest  in  protective  services,  and  interest  in 
leadership. 

Validity  ratings  of  the  31  attributes  for  the  five  job  areas  closely 
followed  the  expected  pattern.  Nine  attributes  met  the  maximum  difference 
cut  for  Core  Technical  Proficiency:  mechanical  comprehension,  movement 
judgment,  physical  strength,  physical  endurance,  balance  and  flexibility, 
involvement  in  athletics,  interest  in  using  tools  and  machines,  interest  in 
rugged  activities,  and  interest  in  protective  services.  No  attributes  met 
the  maximum  difference  cut  for  General  Soldiering,  Effort  and  Leadership, 
or  Personal  Discipline.  Two  attributes  met  the  maximum  difference  cut  for 
Physical  Fitness/Military  Bearing:  work  orientation,  and  interest  in 
efficiency  and  organization. 

SMEs  also  rank  ordered  the  31  attributes  in  terms  of  importance  or 
validity  for  overall  job  performance.  The  mean  ranks  are  shown  in  Tables 
55  and  61,  respectively.  The  five  attributes  given  the  highest  mean 
importance  rankings  for  IIB  were  physical  endurance,  physical  strength, 
verbal  ability,  reasoning,  and  mental  information  processing  (in  order). 

For  63B  they  were  mechanical  comprehension,  interest  in  using  tools  and 
machines,  interest  in  technical  activities,  hand  and  finger  dexterity,  and 
work  orientation.  For  71L  the  five  highest  were  verbal  ability,  reasoning, 
mental  information  processing,  work  orientation,  and  memory. 

The  five  attributes  given  the  highest  mean  validity  rankings  for  IIB 
were  physical  endurance,  physical  strength,  reasoning,  memory  and  mental 
information  processing.  For  63B,  the  top  five  mean  validity  rankings  went 
to  reasoning,  mechanical  comprehension,  mental  information  processing. 
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interest  in  using  tools  and  machines,  and  memory.  The  five  attributes 
given  the  highest  mean  validity  rankings  for  71L  were  verbal  ability,  hand 
and  finger  dexterity,  reasoning,  mental  information  processing,  and  memory. 

For  IIB  and  71L,  the  top  five  validity  rankings  closely  parallel  the 
top  five  rankings  on  importance.  The  63B  validity  and  importance  rankings 
are  dissimilar,  however.  No  ready  explanation  for  the  63B  differences 
presents  itself. 


Conclusions:  Attributes 


We  were  pleased  by  the  results  of  the  pilot  test.  On  the  whole,  we 
think  the  results  show  that  Army  personnel  understand  the  instructions  and 
other  materials  used  in  the  three  rating  tasks  (importance  ratings  of 
attributes  for  the  five  job  performance  areas,  validity  ratings  of 
attributes  for  the  five  job  performance  areas,  and  rankings  of  validity  or 
importance  of  attributes  for  overall  job  performance).  They  can  use  these 
materials  to  make  the  necessary  judgments  and  they  agree  on  those  judgments 
to  a  practically  useful  degree.  The  pattern  of  the  judged  relationships 
between  the  attributes  and  the  five  job  performance  areas,  as  revealed  in 
the  various  analyses  detailed  above,  corresponds  strikingly  to  the  pattern 
that  we  expected  to  find  and  to  the  pattern  found  in  Project  A  research. 


also  hoped  that  the  pilot  test  might  point  clearly  to  either 
the  validity  or  the  importance  methods  as  the  preferred  method.  Alas,  we 
did  not  obtain  such  clear-cut  results.  Both  methods  produced  reliable 

methods  converged  on  similar  results.  We  did  not  achieve 
sufficient  N  s  to  make  some  of  the  comparisons  between  methods  that  were 
planned,  but  we  think  that  the  results  we  do  have  strongly  indicate  that 
either  method  will  produce  meaningful,  reliable  results.  Therefore,  we 
have  opted  to  pursue  the  use  of  the  validity  method  in  Phase  I,  primarily 
because  we  also  will  use  the  validity  method  when  we  collect  judgments  from 
psychologists  (linking  the  attributes  to  task  categories  and  job^ 

seems  best  to  use  the  same  method  for  both  sets  of  judges 
so  that  when  it  comes  time  to  compare  the  two  sets  of  judgments  we  will 
have  removed  at  least  one  source  of  ambiguity  in  the  comparison. 


T  b  IS®  judges  can  make  reliable  judgments  that 

link  the  31  attributes  to  the  five  job  performance  areas.  We  do  not  have 
judgment  data  showing  how  well  they  can  link  the  31  attributes  to  more 
specific  entities--such  as  the  task  categories  or  job  activities.  This  is 
the  judgment  that  we  will  ask  the  psychologists  to  make.  Therefore,  we 
think  It  would  be  prudent  to  ask  Army  judges  to  link  the  attributes  to  a 
subset  of  these  more  specific  job  descriptors  during  Phase  I. 


For  Phase  I  data  collection  then,  we  will  eliminate  the  collection  of 
importance  ratings  and  rankings,  and  use  only  the  validity  method 
Improved  instructions,  including  a  standard  script,  will  be  prepared.  The 
validity  ratings  will  be  collected  for  two  of  the  five  job  performance 
areas.  Core  Technical  and  General  Soldiering,  but  not  for  the  other  three 
areas.  Instead,  five  "common"  job  task  category  items  (rated  most 
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important  across  the  three  MOS)  and  five  "specific"  job  task  category  items 
(rated  most  important  within  the  MOS,  but  not  "common")  will  be  used. 

General  Conclusions 

The  results  for  all  three  job  component  models  show  that  Army 
personnel  understood  the  procedures  and  materials  for  judging  the 
importance  and/or  validity  of  individual  component  for  job  performance. 

The  results  also  show  that  there  was  good  agreement  among  the  judges  on 
those  judgments.  However,  there  was  no  conclusive  evidence  for  favoring 
one  job  component  model  over  the  others. 

Specific  changes  to  each  job  component  model  data  collection  protocol 
have  been  discussed.  We  will  also  be  making  several  changes  in  all  three 
instruments  primarily  to  standardize  the  way  terms  are  used  across  all 
three  exercises.  We  will: 

§  provide  a  consistent  description  of  the  target  job;  judges  will  be 
instructed  to  focus  on  soldiers  with  18  months  on-the-job 
experience  beyond  Basic  Training  and  Advanced  Individual  Training 
and  told  to  consider  the  range  of  possible  job  duties  within  the 
MOS  when  making  their  judgments 

•  improve  instructions,  including  better  examples,  for  all  three 
protocols. 
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Table  la  Number  of  Participants  by  Rank  and  MOS 


Table  2a 

Task  Category  Importance,  Frequency,  and  Difficulty  Rating  Descriptive  Statistics 

For  IIB:  Infantryman 


Task  Category  M 
Perform  operator  maintenance  checks  an60 
Perform  operator  checks  and  services  o60 


Importance  Frequency 


10 

11 


Inspect  mechanical  systems 
Repair  weapons 
Repair  mechanical  systems 
Troubleshoot  mechanical  systems 
Troubleshoot  weapons 

8  Install  electronic  components 

9  Inspect  electrical  systems 
Inspect  electronic  systems 
Repair  electrical  systems 

12  Repair  electronic  components 

13  Troubleshoot  electrical  systems 

14  Troubleshoot  electronic  components 

15  Pack  and  load  materials 

16  Prepare  parachutes 

17  Prepare  equipment  and  supplies  for  ai 

18  Operate  power  excavating  equipment 

19  Operate  wheeled  vehicles 

20  Operate  track  vehicles 

21  Operate  boats 

22  Operate  lifting,  loading,  and  grading 

23  Paint 

24  Install  wire  and  cables 

25  Repair  plastic  and  fiberglass 

26  Repair  metal 

27  Assemble  steel  structures 

28  Install  pipe  assemblies 

29  Construct  wooden  buildings  and  other 

30  Construct  masonry  buildings  and  struc 

31  Operate  gas  and  electric  powered  equi 

32  Select,  layout  and  clean  medical  or  d 

33  Use  audiovisual  equipment 

34  Reproduce  printed  material 

35  Operate  electronic  equipment 

36  Operate  radar 

37  Operate  computer  hardware 

38  Cook 

39  Perform  medical  laboratory  procedures 

40  Conduct  1  and  surveys 

41  Provide  medical  or  dental  treatment 

42  Sketch  maps,  overlays,  or  range  cards 

43  Produce  technical  drawings 

44  Draw  maps  and  overlays 

45  Draw  illustrations 

46  Type 

47  Prepare  technical  forms  and  documents 
Record,  file,  and  dispatch  informatio 
Receive,  store,  and  issue  supplies,  e 

50  Use  hand  and  arm  signals 

51  Read  technical  manuals,  field  manuals 

52  Use  maps 


48 

49 


60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

60 

59 

60 
60 


MEAN 

3.82 

4.63 
1.73 
1.47 

1.63 
1.10 
2.43 
3.10 
1.07 
0.95 
0.82 
0.47 
0.80 
0.52 
2.38 
0.53 
0.93 
0.30 
3.42 
3.78 
1.03 
0.12 
1.25 
1.92 
0.13 
0.20 
0.20 
0.02 
0.25 
0.17 
0.97 
0.00 
0.67 
0.52 
2.27 
0.17 
0.47 
0.32 
0.00 
1.53 
0.42 
3.97 
0.22 
0.80 
1.63 
0.53 
1.07 
0.90 
0.82 
3.95 
3.72 
4.33 


S.D. 

1.62 

0.69 

2.06 

2.00 

2.06 

1.87 

2.24 
1.86 
1.78 
1.68 
1.58 

1.24 

1.53 
1.31 
1.80 
1.42 
1.64 
1.15 

1.54 
1.45 
1.72 
0.58 
1.34 
1.80 
0.70 
0.94 
0.86 
0.13 
0.84 
0.78 
1.40 
0.00 

1.36 
1.02 
1.94 
0.91 
1.21 
1.05 
0.00 
2.13 
1.39 
1.44 
0.90 
1.69 
1.85 
1.17 
1.59 
1.56 
1.61 
1.28 

1.37 

1.10 


N 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 

59 

60 
60 
60 
60 

59 

60 
60 


MEAN 

2.22 

2.77 

0.93 

0.58 

0.77 

0.53 

1.27 

1.63 

0.53 

0.47 

0.38 

0.22 

0.43 

0.28 

1.15 

0.25 

0.40 

0.17 

1.83 

2.07 

0.43 

0.07 

0.98 

0.98 

0.12 

0.12 

0.12 

0.02 

0.20 

0.08 

0.55 

0.00 

0.42 

0.43 

1.23 

0.08 

0.27 

0.18 

0.00 

0.83 

0.18 

2.08 

0.12 

0.42 

0.80 

0.33 

0.63 

0.48 

0.48 

2.29 

2.12 

2.27 


S.D. 

1.01 

0.50 

1.16 

0.85 

1.01 

0.93 

1.33 

1.06 

0.95 

0.87 

0.80 

0.61 

0.89 

0.74 

0.94 

0.70 

0.72 

0.67 

0.91 

0.92 

0.74 

0.31 

0.95 

0.95 

0.56 

0.56 

0.49 

0.13 

0.63 

0.42 

0.81 

0.00 

0.79 

0.85 

1.11 

0.46 

0.69 

0.60 

0.00 

1.20 

0.65 

0.91 

0.49 

0.93 

1.00 

0.71 

0.99 

0.83 

0.97 

0.89 

0.94 

0.86 


Difficulty 


N 

MEAN 

S.D. 

52 

1.52 

0.58 

59 

1.39 

0.53 

26 

1.85 

0.67 

22 

1.86 

0.77 

24 

1.96 

0.55 

17 

2.06 

0.56 

34 

1.82 

0.76 

46 

1.52 

0.55 

17 

2.00 

0.61 

16 

2.00 

0.63 

14 

1.93 

0.73 

8 

2.25 

0.71 

14 

2.00 

0.78 

9 

2.33 

0.50 

43 

1.49 

0.63 

8 

2.38 

0.52 

17 

2.12 

0.78 

4 

1.75 

0.50 

55 

1.35 

0.52 

56 

1.52 

0.54 

19 

1.47 

0.61 

3 

1.33 

0.58 

40 

1.07 

0.27 

37 

1.43 

0.60 

3 

2.00 

0.00 

3 

2.00 

1.00 

4 

2.00 

0.82 

1 

1.00 

. 

7 

1.29 

0.49 

3 

1.33 

0.58 

24 

1.58 

0.58 

0 

17 

1.59 

o!62 

16 

1.13 

0.34 

38 

1.79 

0.78 

2 

3.00 

0.00 

10 

2.60 

0.52 

6 

1.33 

0.52 

0 

22 

2.05 

0^72 

5 

1.40 

0.55 

55 

1.80 

0.65 

4 

1.75 

0.96 

12 

1.92 

0.79 

29 

1.76 

0.79 

13 

1.85 

0.69 

22 

1.86 

0.83 

18 

1.89 

0.68 

14 

1.93 

0.83 

56 

1.23 

0.47 

56 

1.50 

0.63 

58 

2.02 

0.66 

Table  2a  continued 

Task  Category  Importance,  Frequency,  and  Difficulty  Rating  Descriptive  Statistics 

For  IIB:  Infantryman 


Task  Category  N 

53  Send  and  receive  radio  messages  60 

54  Give  oral  reports  60 

55  Receive  clients,  patients,  guests  60 

56  Give  directions  and  instructions  60 

57  Write  documents  and  correspondence  60 

58  Write  and  deliver  presentations  60 

59  Interview  60 

60  Provide  counseling  and  other  interper  60 

61  Decode  data  60 

62  Analyze  electronic  signals  60 

63  Analyze  weather  conditions  60 

64  Order  equipment  and  supplies  60 

65  Estimate  time  and  cost  of  maintenance  60 

66  Plan  placement  or  use  of  tactical  equ  60 

67  Translate  foreign  languages  60 

68  Analyze  intelligence  data  60 

69  Control  money  60 

70  Determine  firing  data  for  indirect  fi  60 

71  Compute  statistics  or  other  mathemati  60 

72  Provide  programming  and  data  process!  60 

73  Control  air  traffic  60 

74  Use  hand  grenades  60 

75  Protect  against  NBC  hazards  60 

76  Handle  demolitions  or  mines  60 

77  Engage  in  hand-to-hand  combat  60 

78  Fire  individual  weapons  60 

79  Control  individuals  and  crowds  60 

80  Customs  and  1 aws  of  war  60 

81  Navigate  60 

82  Survive  in  the  field  60 


83  Load  and  unload  field  artillery  or  ta  60 

84  Fire  heavy  direct  fire  weapons  (e.g.,  60 

85  Prepare  heavy  weapons  for  tactical  us  60 

86  Place  and  camouflage  tactical  equipme  60 

87  Fire  indirect  fire  weapons  (e.g.,  fie  60 


88  Give  first  aid  60 

89  Detect  and  identify  targets  60 

90  Plan,  organize,  monitor  60 

91  Clarify  roles,  provide  feedback  60 

92  Provide  information  60 

93  Recognize,  reward  60 

94  Support  60 

95  Train,  develop  60 

96  Discipline,  punish  60 

97  Act  as  model  60 

98  Conduct  tactical  operations  60 


portar 

ice 

Frequency 

Difficulty 

MEAN 

S.D. 

N 

MEAN 

S.D. 

N 

MEAN 

S.D. 

4.28 

0.87 

60 

2.18 

0.70 

60 

1.75 

0.57 

4.32 

0.98 

60 

2.35 

0.71 

59 

1.56 

0.65 

0.17 

0.91 

60 

0.08 

0.46 

2 

1.50 

0.71 

3.07 

1.74 

60 

1.70 

1.05 

49 

1.51 

0.62 

0.58 

1.31 

60 

0.30 

0.72 

11 

2.45 

0.82 

0.28 

1.03 

60 

0.13 

0.47 

5 

2.40 

0.89 

0.65 

1.45 

60 

0.27 

0.63 

11 

1.64 

0.50 

1.17 

1.93 

60 

0.60 

1.06 

17 

2.18 

0.81 

3.30 

1.83 

60 

1.40 

0.94 

48 

2.10 

0.81 

0.43 

1.25 

60 

0.17 

0.49 

8 

2.25 

0.71 

0.65 

1.41 

60 

0.33 

0.73 

13 

1.69 

0.75 

1.02 

1.70 

60 

0.50 

0.81 

19 

1.68 

0.75 

0.20 

0.94 

60 

0.13 

0.60 

3 

2.00 

1.00 

1.72 

2.19 

60 

0.80 

1.13 

24 

2.13 

0.80 

0.28 

1.09 

60 

0.10 

0.44 

4 

3.00 

0.00 

0.57 

1.51 

60 

0.20 

0.58 

8 

2.50 

0.76 

0.33 

1.11 

60 

0.20 

0.68 

6 

1.83 

0.98 

0.90 

1.71 

60 

0.42 

0.85 

14 

2.07 

0.73 

0.27 

0.95 

60 

0.15 

0.61 

5 

2.00 

1.00 

0.02 

0.13 

60 

0.02 

0.13 

1 

1.00 

0.27 

1.04 

60 

0.07 

0.25 

4 

2.75 

o.'so 

4.37 

1.07 

60 

1.80 

0.86 

58 

1.28 

0.49 

4.65 

0.61 

60 

2.23 

0.72 

60 

1.70 

0.70 

3.83 

1.65 

60 

1.52 

0.85 

53 

2.02 

0.72 

4.05 

1.06 

60 

1.58 

0.79 

59 

1.61 

0.70 

4.87 

0.43 

60 

2.48 

0.72 

60 

1.58 

0.67 

3.05 

1.82 

60 

1.27 

0.88 

49 

1.53 

0.68 

3.77 

1.41 

60 

1.58 

0.87 

57 

1.54 

0.63 

4.37 

1.22 

60 

2.08 

0.87 

57 

2.07 

0.73 

4.73 

0.52 

60 

2.52 

0.68 

60 

1.57 

0.70 

0.53 

1.50 

60 

0.28 

0.83 

7 

1.57 

0.79 

1.87 

2.17 

60 

0.87 

1.10 

27 

2.00 

0.68 

0.75 

1.64 

60 

0.38 

0.85 

11 

2.09 

0.83 

3.82 

1.58 

60 

1.83 

0.99 

54 

1.69 

0.72 

0.62 

1.45 

60 

0.32 

0.77 

10 

2.30 

0.67 

4.52 

0.68 

60 

2.02 

0.72 

60 

1.68 

0.62 

4.30 

1.08 

60 

2.12 

0.88 

58 

1.69 

0.65 

1.38 

1.91 

60 

0.63 

0.92 

23 

1.91 

0.73 

1.40 

1.95 

60 

0.80 

1.22 

22 

1.95 

0.72 

2.52 

2.12 

60 

1.30 

1.20 

38 

1.71 

0.65 

1.03 

1.71 

60 

0.58 

1.00 

18 

1.94 

0.80 

1.60 

2.00 

60 

0.87 

1.16 

26 

1.85 

0.73 

1.25 

1.96 

60 

0.65 

1.05 

19 

2.16 

0.76 

0.97 

1.67 

60 

0.50 

0.91 

16 

2.00 

0.63 

2.92 

2.12 

60 

1.78 

1.33 

42 

2.02 

0.81 

1.22 

1.98 

60 

0.55 

0.91 

18 

2.50 

0.62 

Table  2b 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 


Task  Category 


Inspect  mechanical  systems 
Repair  weapons 
Repair  mechanical  systems 
Troubleshoot  mechanical  systems 
Troubleshoot  weapons 
Install  electronic  components 
Inspect  electrical  systems 
Inspect  electronic  systems 
Repair  electrical  systems 
Repair  electronic  components 
Troubleshoot  electrical  systems 

14  Troubleshoot  electronic  components 

15  Pack  and  load  materials 

16  Prepare  parachutes 

17  Prepare  equipment  and  supplies  for  a 

18  Operate  power  excavating  equipment 

19  Operate  wheeled  vehicles 

20  Operate  track  vehicles 

21  Operate  boats 

22  Operate  lifting,  loading,  and  gradinc 

23  Paint 

24  Install  wire  and  cables 

25  Repair  plastic  and  fiberglass 

26  Repair  metal 

27  Assemble  steel  structures 

28  Install  pipe  assemblies 

29  Construct  wooden  buildings  and  other 

30  Construct  masonry  buildings  and  struc 

31  Operate  gas  and  electric  powered  equi 

32  Select,  layout  and  clean  medical  or  d 

33  Use  audiovisual  equipment 

34  Reproduce  printed  material 

35  Operate  electronic  equipment 

36  Operate  radar 

37  Operate  computer  hardware 

38  Cook 

39  Perform  medical  laboratory  procedures 

40  Conduct  land  surveys 

41  Provide  medical  or  dental  treatment 

42  Sketch  maps,  overlays,  or  range  cards 

43  Produce  technical  drawings 

44  Draw  maps  and  overlays 

45  Draw  illustrations 

46  Type 

47  Prepare  technical  forms  and  documents 

48  Record,  file,  and  dispatch  informatio 

49  Receive,  store,  and  issue  supplies,  e 

50  Use  hand  and  arm  signals 

51  Read  technical  manuals,  field  manuals 

52  Use  maps 


sncy, 

and  Difficulty  Rating  Descriptive 

Statistics 

Vehicie/Generator  Mechanic 

Importance 

Frequency 

Difficulty 

N 

MEAN 

S.D. 

N 

MEAN 

S.D. 

N 

MEAN  S.D. 

an35 

4.69 

0.58 

35 

2.63 

0.60 

35 

1.54  0.66 

o35 

3.63 

1.57 

35 

1.63 

0.91 

33 

1.45  0.56 

35 

4.46 

1.22 

35 

2.43 

0.85 

33 

1.88  0.70 

35 

0.40 

1.19 

35 

0.14 

0.43 

4 

2.00  0.82 

35 

4.46 

1.22 

35 

2.57 

0.81 

33 

1.76  0.66 

35 

4.43 

1.27 

34 

2.29 

0.87 

32 

2.13  0.66 

35 

0.63 

1.33 

35 

0.29 

0.57 

8 

1.88  0.83 

35 

1.03 

1.65 

35 

0.46 

0.78 

11 

1.91  0.54 

35 

4.14 

1.24 

35 

2.17 

0.82 

33 

2.33  0.65 

35 

0.69 

1.55 

35 

0.37 

0.84 

6 

2.50  0.55 

35 

4.17 

0.95 

35 

2.11 

0.68 

35 

1.97  0.66 

35 

1.83 

1.99 

35 

1.00 

1.11 

17 

2.06  0.75 

35 

4.09 

1.29 

35 

2.17 

0.86 

33 

2.24  0.71 

35 

1.37 

2.03 

35 

0.77 

1.19 

12 

2.17  0.72 

35 

2.06 

1.63 

35 

1.11 

0.93 

26 

1.46  0.58 

35 

0.00 

0.00 

35 

0.00 

0.00 

0 

i  35 

0.14 

0.49 

35 

0.09 

0.28 

3 

2.33  o!58 

35 

0.09 

0.51 

35 

0.06 

0.34 

1 

1.00 

35 

4.31 

0.87 

35 

2.51 

0.74 

35 

1.37  0.60 

35 

1.77 

1.96 

35 

1.11 

1.25 

18 

1.56  0.51 

35 

0.00 

0.00 

35 

0.00 

0.00 

0 

3  35 

1.31 

1.81 

35 

0.86 

1.19 

15 

1.67  o!49 

35 

2.09 

1.56 

35 

1.34 

0.87 

30 

1.10  0.40 

35 

0.71 

1.15 

35 

0.46 

0.70 

12 

1.25  0.62 

35 

0.20 

0.47 

35 

0.23 

0.60 

6 

2.33  0.82 

35 

1.51 

1.54 

35 

0.80 

0.87 

20 

1.90  0.85 

35 

0.00 

0.00 

35 

0.00 

0.00 

0 

35 

0.11 

0.53 

35 

0.09 

0.37 

2 

1.50  o!71 

35 

0.06 

0.24 

35 

0.11 

0.53 

2 

1.00  0.00 

i  35 

0.00 

0.00 

35 

0.00 

0.00 

0 

i  35 

2.60 

1.68 

35 

1.34 

0.97 

28 

1.57  0.63 

i  35 

0.00 

0.00 

35 

0.00 

0.00 

0 

35 

0.26 

0.85 

35 

0.14 

0.43 

4 

1.00  0.00 

35 

0.11 

0.32 

35 

0.11 

0.32 

4 

1.00  0.00 

34 

0.65 

1.10 

34 

0.29 

0.46 

10 

2.10  0.57 

35 

0.00 

0.00 

35 

0.00 

0.00 

0 

35 

0.40 

0.98 

35 

0.26 

0.66 

6 

1.83  0.75 

35 

0.00 

0.00 

35 

0.00 

0.00 

0 

35 

0.00 

0.00 

35 

0.00 

0.00 

0 

35 

0.51 

1.27 

35 

0.26 

0.56 

7 

2.00  0.82 

35 

0.14 

0.85 

35 

0.03 

0.17 

1 

2.00 

35 

1.34 

1.70 

35 

0.57 

0.70 

16 

1.75  0.68 

35 

0.00 

0.00 

35 

0.00 

0.00 

0 

35 

0.03 

0.17 

35 

0.03 

0.17 

1 

i!6o 

35 

0.26 

0.61 

35 

0.17 

0.38 

6 

1.50  0.55 

35 

0.37 

0.88 

35 

0.20 

0.47 

6 

1.50  0.55 

35 

2.14 

2.05 

35 

1.37 

1.31 

20 

1.75  0.64 

35 

1.54 

1.80 

35 

0.83 

1.04 

17 

1.65  0.70 

35 

1.06 

1.63 

35 

0.60 

0.95 

13 

1.46  0.52 

35 

2.83 

1.72 

35 

1.57 

1.04 

30 

1.30  0.60 

35 

4.80 

0.53 

35 

2.69 

0.58 

35 

1.83  0.66 

35 

3.14 

1.44 

35 

1.31 

0.83 

32 

1.81  0.74 

Table  2b  continued 

Task  Category  Importance,  Frequency,  and  Difficulty  Rating  Descriptive  Statistics 

For  63B;  Vehicle/Generator  Mechanic 


Task  Category  N 

53  Send  and  receive  radio  messages  35 

54  Give  oral  reports  35 

55  Receive  clients,  patients,  guests  35 

56  Give  directions  and  instructions  35 

57  Write  documents  and  correspondence  35 

58  Write  and  deliver  presentations  35 

59  Interview  35 

60  Provide  counseling  and  other  interper  35 

61  Decode  data  35 

62  Analyze  electronic  signals  35 

63  Analyze  weather  conditions  35 

64  Order  equipment  and  supplies  35 

65  Estimate  time  and  cost  of  maintenance  35 

66  Plan  placement  or  use  of  tactical  equ  35 

67  Translate  foreign  languages  35 

68  Analyze  intelligence  data  35 

69  Control  money  35 

70  Determine  firing  data  for  indirect  fi  35 

71  Compute  statistics  or  other  mathemati  35 

72  Provide  programming  and  data  processi  35 

73  Control  air  traffic  35 

74  Use  hand  grenades  35 

75  Protect  against  NBC  hazards  35 

76  Handle  demolitions  or  mines  35 

77  Engage  in  hand-to-hand  combat  35 

78  Fire  individual  weapons  35 

79  Control  individuals  and  crowds  35 

80  Customs  and  1 aws  of  war  35 

81  Navigate  35 

82  Survive  in  the  field  35 


83  Load  and  unload  field  artillery  or  ta  35 

84  Fire  heavy  direct  fire  weapons  (e.g.,  35 

85  Prepare  heavy  weapons  for  tactical  us  35 

86  Place  and  camouflage  tactical  equipme  35 

87  Fire  indirect  fire  weapons  (e.g.,  fie  35 


88  Give  first  aid  35 

89  Detect  and  identify  targets  35 

90  Plan,  organize,  monitor  35 

91  Clarify  roles,  provide  feedback  35 

92  Provide  information  35 

93  Recognize,  reward  35 

94  Support  35 

95  Train,  develop  35 

96  Discipline,  punish  35 

97  Act  as-  model  35 

98  Conduct  tactical  operations  35 


Importance 

Frequency 

Difficul 

Ity 

MEAN 

S.D. 

N 

MEAN 

S.D. 

N 

MEAN 

s 

.D. 

2.11 

1.81 

35 

1.06 

0.97 

24 

1.92 

0 

72 

2.57 

1.63 

35 

1.17 

0.79 

30 

1.53 

0 

73 

0.03 

0.17 

35 

0.03 

0.17 

1 

3.00 

2.91 

1.38 

35 

1.63 

0.84 

32 

1.28 

0 

52 

0.09 

0.51 

35 

0.06 

0.34 

1 

2.00 

0.00 

0.00 

35 

0.00 

0.00 

0 

0.51 

1.34 

35 

0.31 

0.83 

5 

L40 

0 

55 

0.09 

0.51 

35 

0.06 

0.34 

1 

2.00 

0.54 

1.34 

35 

0.26 

0.66 

6 

2.33 

0 

82 

0.00 

0.00 

35 

0.00 

0.00 

0 

0.09 

0.37 

35 

0.06 

0.24 

2 

i!oo 

0 

00 

1.91 

2.02 

35 

0.97 

1.10 

18 

1.67 

0 

59 

0.51 

1.22 

35 

0.26 

0.61 

7 

1.57 

0 

79 

0.91 

1.65 

35 

0.40 

0.74 

10 

1.80 

0 

79 

0.00 

0.00 

35 

0.00 

0.00 

0 

0.00 

0.00 

35 

0.00 

0.00 

0 

0.00 

0.00 

35 

0.00 

0.00 

0 

0.14 

0.55 

35 

0.09 

0.28 

3 

1.'33 

0 

58 

0.17 

0.62 

35 

0.11 

0.40 

3 

2.00 

1 

00 

0.06 

0.34 

35 

0.03 

0.17 

1 

1.00 

0.00 

0.00 

35 

0.00 

0.00 

0 

2.74 

1.67 

35 

1.17 

0.79 

31 

i!29 

0 

53 

4.09 

1.20 

35 

1.69 

0.87 

35 

1.69 

0. 

72 

1.20 

1.69 

35 

0.49 

0.66 

14 

1.57 

0, 

2.40 

1.79 

35 

1.06 

0.91 

26 

1.58 

0. 

3.97 

1.10 

35 

1.60 

0.69 

35 

1.46 

0. 

1.37 

1.61 

35 

0.77 

0.91 

19 

1.53 

0. 

2.86 

1.57 

35 

1.17 

0.79 

31 

1.55 

0. 

68 

2.94 

1.59 

35 

1.26 

0.89 

30 

1.77 

0. 

82 

3.69 

1.59 

35 

1.51 

0.92 

32 

1.50 

0. 

67 

0.00 

0.00 

35 

0.00 

0.00 

0 

0.14 

0.85 

35 

0.03 

0.17 

1 

2. *00 

0.00 

0.00 

35 

0.00 

0.00 

0 

2.80 

1.83 

35 

1.31 

0.99 

27 

K41 

0! 

64 

0.00 

0.00 

35 

0.00 

0.00 

0 

3.80 

1.28 

35 

1.54 

0.92 

34 

i!68 

0! 

68 

2.54 

1.72 

35 

1.09 

0.85 

27 

1.70 

0. 

78 

1.14 

1.75 

35 

0.51 

0.82 

12 

2.08 

0. 

79 

0.77 

1.50 

35 

0.37 

0.77 

8 

2.25 

0. 

89 

1.94 

1.89 

35 

0.97 

1.01 

20 

1.55 

0. 

69 

0.71 

1.32 

35 

0.34 

0.68 

9 

1.67 

0. 

87 

1.17 

1.67 

35 

0.51 

0.78 

13 

1.85 

0. 

80 

0.94 

1.61 

35 

0.46 

0.82 

10 

2.00 

0. 

82 

0.60 

1.29 

35 

0.29 

0.67 

7 

2.14 

0. 

90 

3.40 

1.59 

35 

2.20 

1.08 

31 

1.71 

0. 

74 

0.43 

1.14 

35 

0.23 

0.65 

5 

2.20 

0. 

84 

Table  2c 

Task  Category  Importance,  Frequency,  and  Difficulty  Rating  Descriptive  Statistics 

For  TIL?  Ariniinietv<9^^ ua 


Task  Category 


2 

3 

4 

5 

6 

7 

8 
9 

10 

11 


Inspect  mechanical  systems 
Repair  weapons 
Repair  mechanical  systems 
Troubleshoot  mechanical  systems 
Troubleshoot  weapons 
Install  electronic  components 
Inspect  electrical  systems 
Inspect  electronic  systems 
Repair  electrical  systems 

12  Repair  electronic  components 

13  Troubleshoot  electrical  systems 

14  Troubleshoot  electronic  components 

15  Pack  and  load  materials 

16  Prepare  parachutes 

17  Prepare  equipment  and  supplies  for  a 

18  Operate  power  excavating  equipment 

19  Operate  wheeled  vehicles 

20  Operate  track  vehicles 

21  Operate  boats 

22  Operate  lifting,  loading,  and  grading 

23  Paint 

24  Install  wire  and  cables 

25  Repair  plastic  and  fiberglass 

26  Repair  metal 

27  Assemble  steel  structures 

28  Install  pipe  assemblies 

29  Construct  wooden  buildings  and  other 

30  Construct  masonry  buildings  and  stru( 

31  Operate  gas  and  electric  powered  equ' 

32  Select,  layout  and  clean  medical  or  d 

33  Use  audiovisual  equipment 

34  Reproduce  printed  material 

35  Operate  electronic  equipment 

36  Operate  radar 

37  Operate  computer  hardware 

38  Cook 

39  Perform  medical  laboratory  procedures 

40  Conduct  1  and  surveys 

41  Provide  medical  or  dental  treatment 

42  Sketch  maps,  overlays,  or  range  cards 

43  Produce  technical  drawings 

44  Draw  maps  and  overlays 

45  Draw  illustrations 

46  Type 

47  Prepare  technical  forms  and  documents 

48  Record,  file,  and  dispatch  informatio 

49  Receive,  store,  and  issue  supplies,  e 

50  Use  hand  and  arm  signals 

51  Read  technical  manuals,  field  manuals 

52  Use  maps 


Importance 

Frequency 

Difficulty 

N 

MEAN 

S.D. 

N 

MEAN 

S.D. 

N 

MEAN 

S.D. 

an39 

1.87 

1.59 

39 

0.77 

0.63 

26 

1.50 

0.58 

o39 

2.77 

1.77 

39 

1.15 

0.78 

32 

1.56 

0.56 

39 

0.33 

0.96 

39 

0.15 

0.43 

5 

1.20 

0.45 

39 

0.21 

0.73 

39 

0.10 

0.38 

3 

2.00 

1.00 

39 

0.08 

0.48 

39 

0.03 

0.16 

1 

1.00 

39 

0.18 

0.79 

39 

0.08 

0.35 

2 

2.00 

0.00 

39 

0.79 

1.56 

39 

0.28 

0.56 

9 

1.89 

0.33 

39 

1.54 

1.73 

39 

0.72 

0.83 

20 

1.85 

0.49 

39 

0.44 

1.12 

39 

0.21 

0.52 

6 

1.67 

0.52 

39 

0.18 

0.79 

39 

0.08 

0.35 

2 

2.00 

0.00 

39 

0.13 

0.80 

39 

0.05 

0.32 

1 

2.00 

39 

0.15 

0.81 

39 

0.08 

0.35 

2 

2.50 

0.71 

39 

0.03 

0.16 

39 

0.03 

0.16 

1 

1.00 

39 

0.00 

0.00 

39 

0.00 

0.00 

0 

39 

0.77 

1.13 

39 

0.46 

0.64 

15 

i!40 

0.51 

39 

0..13 

0.80 

39 

0.08 

0.48 

1 

2.00 

i  39 

0.10 

0.64 

39 

0.05 

0.32 

1 

1.00 

39 

0.10 

0.64 

39 

0.05 

0.32 

1 

1.00 

39 

2.23 

1.35 

39 

1.18 

0.72 

33 

1.24 

O.H 

39 

0.15 

0.71 

39 

0.10 

0.50 

2 

1.50 

0.71 

39 

0.00 

0.00 

39 

0.00 

0.00 

0 

3  39 

0.13 

0.66 

39 

0.10 

0.50 

2 

2.00 

0.00 

39 

0.51 

0.60 

39 

0.46 

0.51 

18 

1.00 

0.00 

39 

0.56 

1.14 

39 

0.28 

0.56 

9 

1.33 

0.50 

39 

0.00 

0.00 

39 

0.00 

0.00 

0 

39 

0.03 

0.16 

39 

0.03 

0.16 

1 

i!oo 

39 

0.00 

0.00 

39 

0.00 

0.00 

0 

39 

0.00 

0.00 

39 

0.00 

0.00 

0 

39 

0.00 

0.00 

39 

0.00 

0.00 

0 

:  39 

0.00 

0.00 

39 

0.00 

0.00 

0 

i  39 

1.13 

1.36 

39 

0.54 

0.55 

20 

i.'oo 

0.83 

1  39 

0.13 

0.80 

39 

0.08 

0.48 

1 

3.00 

39 

0.62 

1.21 

39 

0.36 

0.67 

10 

1.60 

0.97 

39 

3.00 

1.34 

39 

2.10 

1.02 

36 

1.47 

0.61 

39 

2.13 

1.75 

39 

1.23 

1.09 

25 

2.04 

0.54 

39 

0.13 

0.80 

39 

0.08 

0.48 

1 

3.00 

39 

3.74 

1.04 

39 

2.28 

0.79 

38 

2.18 

0.65 

39 

0.15 

0.81 

39 

0.10 

0.50 

2 

1.50 

0.71 

39 

0.03 

0.16 

39 

0.03 

0.16 

1 

1.00 

39 

0.67 

1.24 

39 

0.33 

0.58 

11 

2.09 

0.70 

39 

0.13 

0.80 

39 

0.05 

0.32 

1 

2.00 

39 

1.08 

1.42 

39 

0.51 

0.64 

17 

1.82 

0.73 

39 

0.13 

0.52 

39 

0.10 

0.38 

3 

2.00 

1.00 

39 

0.28 

0.94 

39 

0.15 

0.49 

4 

2.00 

0.82 

39 

0.64 

1.09 

39 

0.41 

0.68 

13 

2.00 

0.71 

39 

4.82 

0.45 

39 

2.95 

0.22 

39 

1.85 

0.63 

39 

4.51 

0.64 

39 

2.74 

0,44 

39 

1.82 

0.64 

39 

4.51 

0.68 

39 

2.79 

0.41 

39 

1.74 

0.64 

39 

3.36 

1.50 

39 

1.92 

0.98 

35 

1.54 

0.56 

39 

1.08 

1.33 

39 

0.62 

0.71 

20 

1.30 

0.47 

39 

3.77 

1.44 

39 

2.23 

0.78 

38 

1.84 

0.72 

39 

2.31 

1.54 

39 

1.08 

0.77 

32 

1.84 

0.72 

Table  2c  continued 

Task  Category  Importance,  Frequency,  and  Difficulty  Rating  Descriptive  Statistics 

For  71L:  Administrative  Specialist 


Task  Category  N 

53  Send  and  receive  radio  messages  39 

54  Give  oral  reports  39 

55  Receive  clients,  patients,  guests  39 

56  Give  directions  and  instructions  39 

57  Write  documents  and  correspondence  39 

58  Write  and  deliver  presentations  39 

59  Interview  39 

60  Provide  counseling  and  other  interper  39 

61  Decode  data  39 

62  Analyze  electronic  signals  39 

63  Analyze  weather  conditions  39 

64  Order  equipment  and  supplies  39 

65  Estimate  time  and  cost  of  maintenance  39 

66  Plan  placement  or  use  of  tactical  equ  39 

67  Translate  foreign  languages  39 

68  Analyze  intelligence  data  39 

69  Control  money  39 

70  Determine  firing  data  for  indirect  fi  39 

71  Compute  statistics  or  other  mathemati  39 

72  Provide  programming  and  data  process!  39 

73  Control  air  traffic  39 

74  Use  hand  grenades  39 

75  Protect  against  NBC  hazards  39 

76  Handle  demolitions  or  mines  39 

77  Engage  in  hand-to-hand  combat  39 

78  Fire  individual  weapons  39 

79  Control  individuals  and  crowds  39 

80  Customs  and  laws  of  war  38 

81  Navigate  39 

82  Survive  in  the  field  39 


83  Load  and  unload  field  artillery  or  ta  39 

84  Fire  heavy  direct  fire  weapons  (e.g.,  39 

85  Prepare  heavy  weapons  for  tactical  us  39 

86  Place  and  camouflage  tactical  equipme  39 

87  Fire  indirect  fire  weapons  (e.g.,  fie  39 


88  Give  first  aid  39 

89  Detect  and  identify  targets  39 

90  Plan,  organize,  monitor  39 

91  Clarify  roles,  provide  feedback  38 

92  Provide  information  38 

93  Recognize,  reward  39 

94  Support  39 

95  Train,  develop  39 

96  Discipline,  punish  39 

97  Act  as  model  39 

98  Conduct  tactical  operations  39 


Importance  Frequency  Difficulty 

MEAN  S.D.  N  MEAN  S.D.  N  MEAN  S.D. 

2.00  1.62  39  0.90  0.75  28  1.71  0.60 

2.56  1.45  39  1.23  0.74  34  1.47  0.51 

2.28  1.78  39  1.54  1.17  28  1.39  0.63 

3.28  1.21  39  2.03  0.84  38  1.58  0.60 

3.21  1.63  39  1.87  0.95  34  2.03  0.63 

0.90  1.29  39  0.44  0.68  14  2.14  0.77 

1.38  1.68  39  0.72  0.94  17  1.65  0.70 

0.74  1.35  39  0.44  0.79  11  1.55  0.69 

0.74  1.37  39  0.33  0.62  11  2.18  0.60 

0.03  0.16  39  0.03  0.16  1  2.00  . 

0.03  0.16  39  0.03  0.16  1  2.00  . 

2.15  1.53  39  1.23  0.93  28  1.32  0.48 

0.13  0.57  39  0.05  0.22  2  3.00  0.00 

0.46  1.00  39  0.26  0.50  9  1.56  0.53 

0.15  0.67  39  0.10  0.38  3  2.33  1.15 

0.21  0.80  39  0.08  0.27  3  1.67  0.58 

0.82  1.65  39  0.41  0.85  9  1.56  0.73 

0.36  1.09  39  0.13  0.34  5  2.00  0.71 

0.79  1.26  39  0.46  0.72  13  2.15  0.69 

0.87  1.49  39  0.56  0.94  12  2.17  0.83 

0.00  0.00  39  0.00  0.00  ’0 

2.03  1.63  39  0.87  0.66  29  1.38  o!49 

3.13  1.69  39  1.08  0.62  35  1.74  0.66 

1.13  1.70  39  0.44  0.64  15  2.00  0.76 

1.72  1.57  39  0.74  0.64  26  1.54  0.65 

3.15  1.73  39  1.10  0.72  34  1.79  0.59 

0.90  1.23  39  0.44  0.55  16  1.50  0.63 

2.16  1.53  38  0.87  0.53  30  1.37  0.56 

2.21  1.64  39  0.87  0.61  30  2.03  0.76 

3.03  1.72  39  1.10  0.64  35  1.74  0.70 

0.15  0.81  39  0.10  0.50  2  2.00  0.00 

0.13  0.66  39  0.10  0.50  2  2.00  0.00 

0.10  0.64  39  0.05  0.32  1  2.00 

1.33  1.66  39  0.56  0.68  18  1.61  o!61 

0.21  0.80  39  0.08  0.27  3  2.33  0.58 

3.18  1.52  39  1.10  0.45  37  1.68  0.58 

1.95  1.56  39  0.85  0.59  29  1.90  0.77 

1.62  1.80  39  0.92  1.04  21  1.62  0.74 

1.13  1.63  38  0.74  1.08  15  1.93  0.80 

1.87  1.71  39  1.15  1.14  24  1.46  0.66 

1.15  1.57  39  0.64  0.90  16  1.50  0.82 

1.72  1.82  39  0.92  1.01  21  1.62  0.74 

1.49  1.78  39  0.77  0.99  19  1.84  0.76 

1.03  1.55  39  0.51  0.79  14  1.93  0.83 

2.92  1.87  39  1.87  1.26  30  1.87  0.82 

0.46  1.21  39  0.26  0.72  6  1.50  0.84 


Table  3a 

Task  Category  Importance  Rating  Descripti 
Sorted  by  Mean  Rating 
For  IIB:  Infantryman 


No. Task  Category  N 

78  Fire  individual  weapons  60 

82  Survive  in  the  field  60 

75  Protect  against  NBC  hazards  60 

2  Perform  operator  checks  and  services  60 

88  Give  first  aid  60 

74  Use  hand  grenades  60 

81  Navigate  60 

52  Use  maps  60 

54  Give  oral  reports  60 

89  Detect  and  identify  targets  60 

53  Send  and  receive  radio  messages  60 

77  Engage  in  hand-to-hand  combat  60 

42  Sketch  maps,  overlays,  or  range  cards  60 

50  Use  hand  and  arm  signals  59 

76  Handle  demolitions  or  mines  60 

1  Perform  operator  maintenance  checks  an60 

86  Place  and  camouflage  tactical  equipme  60 

20  Operate  track  vehicles  60 

80  Customs  and  laws  of  war  60 

51  Read  technical  manuals,  field  manuals  60 

19  Operate  wheeled  vehicles  60 

61  Decode  data  60 

8  Install  electronic  components  60 

56  Give  directions  and  instructions  60 

79  Control  individuals  and  crowds  60 

97  Act  as  model  60 

92  Provide  information  60 

7  Troubleshoot  weapons  60 

15  Pack  and  load  materials  60 

35  Operate  electronic  equipment  60 

24  Install  wire  and  cables  60 

84  Fire  heavy  direct  fire  weapons  (e.g.,  60 
3  Inspect  mechanical  systems  60 

66  Plan  placement  or  use  of  tactical  equ  60 
5  Repair  mechanical  systems  60 

45  Draw  illustrations  60 

94  Support  60 

40  Conduct  land  surveys  60 

4  Repair  weapons  60 

91  Clarify  roles,  provide  feedback  60 

90  Plan,  organize,  monitor  60 

23  Paint  60 

95  Train,  develop  60 

98  Conduct  tactical  operations  60 

60  Provide  counseling  and  other  interper  60 

6  Troubleshoot  mechanical  systems  60 

9  Inspect  electrical  systems  60 

47  Prepare  technical  forms  and  documents  60 

21  Operate  boats  60 

93  Recognize,  reward  60 

64  Order  equipment  and  supplies  60 

31  Operate  gas  and  electric  powered  equi  60 


Statistics 


MEAN  S.D. 

4.87  0.43 
4.73  0.52 
4.65  0.61 

4.63  0.69 

4.52  0.68 
4.37  1.07 

4.37  1.22 

4.33  1.10 

4.32  0.98 

4.30  1.08 

4.28  0.87 
4.05  1.06 

3.97  1.44 
3.95  1.28 

3.83  1.65 

3.82  1.62 
3.82  1.58 
3.78  1.45 
3.77  1.41 

3.72  1.37 

3.42  1.54 

3.30  1.83 

3.10  1.86 

3.07  1.74 
3.05  1,82 

2.92  2.12 

2.52  2.12 

2.43  2.24 

2.38  1.80 

2.27  1.94 

1.92  1.80 

1.87  2.17 

1.73  2.06 
1.72  2.19 

1.63  2.06 

1.63  1.85 

1.60  2.00 

1.53  2.13 
1.47  2.00 
1.40  1.95 

1.38  1.91 

1.25  1.34 
1.25  1.96 
1.22  1.98 
1.17  1.93 

1.10  1.87 

1.07  1.78 

1.07  1.59 
1.03  1.72 
1.03  1.71 
1.02  1.70 
0.97  1.40 


Table  3a  continued 

Task  Category  Importance  Rating  Descripti 
Sorted  by  Mean  Rating 
For  IIB:  Infantryman 


No. Task  Category  N 

96  Discipline,  punish  60 

10  Inspect  electronic  systems  60 

17  Prepare  equipment  and  supplies  for  ai  60 

48  Record,  file,  and  dispatch  informat io  60 

70  Determine  firing  data  for  indirect  fi  60 

11  Repair  electrical  systems  60 

49  Receive,  store,  and  issue  supplies,  e  60 

13  Troubleshoot  electrical  systems  60 

44  Draw  maps  and  overlays  60 

85  Prepare  heavy  weapons  for  tactical  us  60 

33  Use  audiovisual  equipment  60 

59  Interview  60 

63  Analyze  weather  conditions  60 

87  Fire  indirect  fire  weapons  (e.g.,  fie  60 

57  Write  documents  and  correspondence  60 

68  Analyze  intelligence  data  60 

16  Prepare  parachutes  60 

46  Type  60 

83  Load  and  unload  field  artillery  or  ta  60 

14  Troubleshoot  electronic  components  60 

34  Reproduce  printed  material  60 

12  Repair  electronic  components  60 

37  Operate  computer  hardware  60 

62  Analyze  electronic  signals  60 

41  Provide  medical  or  dental  treatment  60 

69  Control  money  60 

38  Cook  60 

18  Operate  power  excavating  equipment  60 

58  Write  and  deliver  presentations  60 

67  Translate  foreign  languages  60 

71  Compute  statistics  or  other  mathemati  60 

73  Control  air  traffic  60 

29  Construct  wooden  buildings  and  other  60 

43  Produce  technical  drawings  60 

26  Repair  metal  60 

27  Assemble  steel  structures  60 

65  Estimate  time  and  cost  of  maintenance  60 

30  Construct  masonry  buildings  and  struc  60 

36  Operate  radar  60 

55  Receive  clients,  patients,  guests  60 
25  Repair  plastic  and  fiberglass  60 

22  Operate  lifting,  loading,  and  grading  60 

28  Install  pipe  assemblies  60 

72  Provide  programming  and  data  process!  60 
32  Select,  layout  and  clean  medical  or  d  60 

39  Perform  medical  laboratory  procedures  60 


Statistics 


MEAN  S.D. 
0.97  1.67 
0.95  1.68 
0.93  1.64 

0.90  1.56 
0.90  1.71 
0.82  1.58 
0.82  1.61 
0.80  1.53 
0.80  1.69 

0.75  1.64 
0.67  1.36 

0.65  1.45 
0.65  1.41 

0.62  1.45 
0.58  1.31 
0.57  1.51 

0.53  1.42 

0.53  1.17 
0.53  1.50 
0.52  1.31 
0.52  1.02 
0.47  1.24 
0.47  1.21 
0.43  1.25 

0.42  1.39 
0.33  1.11 
0.32  1.05 
0.30  1.15 
0.28  1.03 
0.28  1.09 
0.27  0.95 
0.27  1.04 

0.25  0.84 
0.22  0.90 
0.20  0.94 
0.20  0.86 
0.20  0.94 
0.17  0.78 
0.17  0.91 
0.17  0.91 
0.13  0.70 
0.12  0.58 
0.02  0.13 
0.02  0.13 
0.00  0.00 
0.00  0.00 


Table  3b 

Task  Category  Importance  Rating  Descriptive  Statistics 


Sorted  by  Mean  Rating 
For  63B:  Vehicle/Generator  Mechanic 


No. Task  Category 

N 

MEAN 

S.D. 

51  Read  technical  manuals,  field  manuals 

35 

4.80 

0.53 

1  Perform  operator  maintenance  checks  an35 

4.69 

0.58 

3  Inspect  mechanical  systems 

35 

4.46 

1.22 

5  Repair  mechanical  systems 

35 

4.46 

1.22 

6  Troubleshoot  mechanical  systems 

35 

4.43 

1.27 

19  Operate  wheeled  vehicles 

35 

4.31 

0.87 

11  Repair  electrical  systems 

35 

4.17 

0.95 

9  Inspect  electrical  systems 

35 

4.14 

1.24 

13  Troubleshoot  electrical  systems 

35 

4.09 

1.29 

75  Protect  against  NBC  hazards 

35 

4.09 

1.20 

78  Fire  individual  weapons 

35 

3.97 

1.10 

88  Give  first  aid 

35 

3.80 

1.28 

82  Survive  in  the  field 

35 

3.69 

1.59 

2  Perform  operator  checks  and  services  i 

o35 

3.63 

1.57 

97  Act  as  model 

35 

3.40 

1.59 

52  Use  maps 

35 

3.14 

1.44 

81  Navigate 

35 

2.94 

1.59 

56  Give  directions  and  instructions 

35 

2.91 

1.38 

80  Customs  and  laws  of  war 

35 

2.86 

1.57 

50  Use  hand  and  arm  signals 

35 

2.83 

1.72 

86  Place  and  camouflage  tactical  equipme 

35 

2.80 

1.83 

74  Use  hand  grenades 

35 

2.74 

1.67 

31  Operate  gas  and  electric  powered  equi 

35 

2.60 

1.68 

54  Give  oral  reports 

35 

2.57 

1.63 

89  Detect  and  identify  targets 

35 

2.54 

1.72 

77  Engage  in  hand-to-hand  combat 

35 

2.40 

1.79 

47  Prepare  technical  forms  and  documents 

35 

2.14 

2.05 

53  Send  and  receive  radio  messages 

35 

2.11 

1.81 

23  Paint 

35 

2.09 

1.56 

15  Pack  and  load  materials 

35 

2.06 

1.63 

92  Provide  information 

35 

1.94 

1.89 

64  Order  equipment  and  supplies 

35 

1.91 

2.02 

12  Repair  electronic  components 

35 

1.83 

1.99 

20  Operate  track  vehicles 

35 

1.77 

1.96 

48  Record,  file,  and  dispatch  informat io 

35 

1.54 

1.80 

26  Repair  metal 

35 

1.51 

1.54 

14  Troubleshoot  electronic  components 

35 

1.37 

2.03 

79  Control  individuals  and  crowds 

35 

1.37 

1.61 

42  Sketch  maps,  overlays,  or  range  cards 

35 

1.34 

1.70 

22  Operate  lifting,  loading,  and  grading 

35 

1.31 

1.81 

76  Handle  demolitions  or  mines 

35 

1.20 

1.69 

94  Support 

35 

1.17 

1.67 

90  Plan,  organize,  monitor 

35 

1.14 

1.75 

49  Receive,  store,  and  issue  supplies,  e 

35 

1.06 

1.63 

8  Install  electronic  components 

35 

1.03 

1.65 

95  Train,  develop 

35 

0.94 

1.61 

66  Plan  placement  or  use  of  tactical  equ 

35 

0.91 

1.65 

91  Clarify  roles,  provide  feedback 

35 

0.77 

1.50 

24  Install  wire  and  cables 

35 

0.71 

1.15 

93  Recognize,  reward 

35 

0.71 

1.32 

10  Inspect  electronic  systems 

35 

0.69 

1.55 

35  Operate  electronic  equipment 

34 

0.65 

1.10 

Table  3b  continued 

Task  Category  Importance  Rating  Descriptive  Statistics 
Sorted  by  Mean  Rating 
For  63B:  Vehicle/Generato** 

No. Task  Category 
7  Troubleshoot  weapons 
96  Discipline,  punish 
61  Decode  data 
40  Conduct  land  surveys 
59  Interview 

65  Estimate  time  and  cost  of  maintenance  35 
98  Conduct  tactical  operations 
4  Repair  weapons 
37  Operate  computer  hardware 
46  Type 

33  Use  audiovisual  equipment 
45  Draw  illustrations 
25  Repair  plastic  and  fiberglass 


41  Provide  medical  or  dental  treatment 
70  Determine  firing  data  for  indirect  fi 
84  Fire  heavy  direct  fire  weapons  (e.g.,  35 

28  Install  pipe  assemblies 
34  Reproduce  printed  material 
18  Operate  power  excavating  equipment 
57  Write  documents  and  correspondence 
60  Provide  counseling  and  other  interper  35 
63  Analyze  weather  conditions 

29  Construct  wooden  buildings  and  other  _ 

72  Provide  programming  and  data  process i  35 
44  Draw  maps  and  overlays 
55  Receive  clients,  patients,  guests 
16  Prepare  parachutes 
21  Operate  boats 
27  Assemble  steel  structures 


36  Operate  radar 

38  Cook 

39  Perform  medical  laboratory  procedures  35 
43  Produce  technical  drawings 
58  Write  and  deliver  presentations 
62  Analyze  electronic  signals 

67  Translate  foreign  languages 

68  Analyze  intelligence  data 

69  Control  money 
73  Control  air  traffic 


N 

MEAN 

S.D. 

35 

0.63 

1.33 

35 

0.60 

1.29 

35 

0.54 

1.34 

35 

0.51 

1.27 

35 

0.51 

1.34 

35 

0.51 

1.22 

35 

0.43 

1.14 

35 

0.40 

1.19 

35 

0.40 

0.98 

35 

0.37 

0.88 

35 

0.26 

0.85 

35 

0.26 

0.61 

35 

0.20 

0.47 

35 

0.17 

0.62 

35 

0.14 

0.49 

35 

0.14 

0.85 

35 

0.14 

0.55 

35 

0.14 

0.85 

35 

0.11 

0.53 

35 

0.11 

0.32 

35 

0.09 

0.51 

35 

0.09 

0.51 

35 

0.09 

0.51 

35 

0.09 

0.37 

35 

0.06 

0.24 

35 

0.06 

0.34 

35 

0.03 

0.17 

35 

0.03 

0.17 

35 

0.00 

0.00 

35 

0.00 

0.00 

35 

0.00 

0.00 

35 

0.00 

0.00 

35 

0.00 

0.00 

35 

0.00 

0.00 

35 

0.00 

0.00 

35 

0.00 

0.00 

35 

0.00 

0.00 

35 

0.00 

0.00 

35 

0.00 

0.00 

35 

0.00 

0.00 

35 

0.00 

0.00 

35 

0.00 

0.00 

35 

0.00 

0.00 

35 

0.00 

0.00 

35 

0.00 

0.00 

35 

0.00 

0.00 

Table  3c 

Task  Category  Importance  Rating  Descriptive  Statistics 
Sorted  by  Mean  Rating 
For  71L:  Administrative  Specialist 
No. Task  Category  N  MEAN 

46  Type  39 

47  Prepare  technical  forms  and  documents  39 

48  Record,  file,  and  dispatch  informatio  39 
51  Read  technical  manuals,  field  manuals  39 
37  Operate  computer  hardware 

49  Receive,  store,  and  issue  supplies, 

56  Give  directions  and  instructions 

57  Write  documents  and  correspondence 


88  Give  first  aid 
78  Fire  individual  weapons 
75  Protect  against  NBC  hazards 
82  Survive  in  the  field 
34  Reproduce  printed  material 
97  Act  as  model 


39 

39 

39 

39 

39 

39 

39 

39 

39 

39 


2  Perform  operator  checks  and  services  o39 

54  Give  oral  reports  39 

52  Use  maps  39 

55  Receive  clients,  patients,  guests  39 

19  Operate  wheeled  vehicles  39 

81  Navigate  39 

80  Customs  and  laws  of  war  38 

64  Order  equipment  and  supplies  39 

35  Operate  electronic  equipment  39 

74  Use  hand  grenades  39 

53  Send  and  receive  radio  messages  39 

89  Detect  and  identify  targets  39 

1  Perform  operator  maintenance  checks  an39 

92  Provide  information  38 

77  Engage  in  hand-to-hand  combat  39 

94  Support  39 

90  Plan,  organize,  monitor  39 

8  Install  electronic  components  39 

95  Train,  develop  39 

59  Interview  39 

86  Place  and  camouflage  tactical  equipme  39 

93  Recognize,  reward  39 

91  Clarify  roles,  provide  feedback  38 

31  Operate  gas  and  electric  powered  equi  39 
76  Handle  demolitions  or  mines  39 

42  Sketch  maps,  overlays,  or  range  cards  39 
50  Use  hand  and  arm  signals  39 

96  Discipline,  punish  39 

58  Write  and  deliver  presentations  39 

79  Control  individuals  and  crowds  39 

72  Provide  programming  and  data  processi  39 
69  Control  money  39 

39 
39 
39 


Troubleshoot  weapons 
71  Compute  statistics  or  other  mathemati 
15  Pack  and  load  materials 

60  Provide  counseling  and  other  interper  39 

61  Decode  data  3g 

40  Conduct  land  surveys  39 


4.82 

4.51 

4.51 

3.77 
3.74 
3.36 

3.28 

3.21 
3.18 

3.15 

3.13 
3.03 
3.00 
2.92 

2.77 
2.56 
2.31 

2.28 
2.23 

2.21 

2.16 

2.15 

2.13 
2.03 
2.00 
1.95 
1.87 
1.87 
1.72 
1.72 
1.62 
1.54 
1.49 
1.38 
1.33 

1.15 

1.13 
1.13 
1.13 
1.08 
1.08 
1.03 
0.90 
0.90 
0.87 
0.82 
0.79 
0.79 
0.77 
0.74 
0.74 
0.67 


S.D. 

0.45 

0.64 

0.68 

1.44 
1.04 
1.50 
1.21 

1.63 

1.52 
1.73 

1.69 

1.72 

1.34 
1.87 

1.77 

1.45 

1.54 

1.78 

1.35 

1.64 

1.53 
1.53 
1.75 
1.63 
1.62 

1.56 
1.59 
1.71 

1.57 
1.82 
1.80 

1.73 
1.78 
1.68 
1.66 
1.57 
1.63 

1.36 

1.70 
1.42 
1.33 

1.55 
1.29 

1.23 
1.49 

1.65 

1.56 
1.26 
1.13 
1.35 

1.37 

1.24 


Table  3c  continued 

Task  Category  Importance  Rating  Descriptive  Statistics 
Sorted  by  Mean  Rating 


For  71L:  Administrative  Specialist 

No. Task  Category 

N 

MEAN 

S.D. 

45  Draw  illustrations 

39 

0.64 

1.09 

33  Use  audiovisual  equipment 

39 

0.62 

1.21 

24  Install  wire  and  cables 

39 

0.56 

1.14 

23  Paint 

39 

0.51 

0.60 

66  Plan  placement  or  use  of  tactical 

equ  39 

0.46 

1.00 

98  Conduct  tactical  operations 

39 

0.46 

1.21 

9  Inspect  electrical  systems 

39 

0.44 

1.12 

70  Determine  firing  data  for  indirect 

fi  39 

0.36 

1.09 

3  Inspect  mechanical  systems 

39 

0.33 

0.96 

44  Draw  maps  and  overlays 

39 

0.28 

0.94 

4  Repair  weapons 

39 

0.21 

0.73 

68  Analyze  Intelligence  data 

39 

0.21 

0.80 

87  Fire  indirect  fire  weapons  (e.g.. 

fie  39 

0.21 

0.80 

6  Troubleshoot  mechanical  systems 

39 

0.18 

0.79 

10  Inspect  electronic  systems 

39 

0.18 

0.79 

12  Repair  electronic  components 

39 

0.15 

0.81 

Table  4a 

Job  Activity  Importance,  Frequency,  and  Difficulty  Rating  Descriptive  Statistics 

IIB  Infantryman 


No. Job  Activity  N 

1  Work  in  a  team  60 

2  Lead  a  team  60 

3  Support/counsel  peers  60 

4  Support/counsel  subordinates  60 

5  Coach  peers  60 

6  Coach  subordinates  59 

7  Make  oral  reports  {to  individuals)  60 

8  Make  oral  reports  {to  groups)  60 

9  Relay  oral  instructions  60 

10  Ask  questions  60 

11  Record  information  60 

12  Write  brief  messages  60 

13  Write  longer  reports  60 

14  Monitor/interpret  verbal  messages  60 

15  Recall  verbal  information  60 

16  Monitor/interpret  numerical  informati  60 

17  Recall  numerical  information  60 

18  Monitor/interpret  figural  information  60 

19  Recall  figural  information  60 

20  Follow  oral  directions  59 

21  Follow  written  directions  60 

22  Judge  size  and  distance  60 

23  Judge  location  60 

24  Judge  paths  of  moving  objects  60 

25  Solve  electrical  system  problems  60 

26  Solve  mechanical  system  problems  60 

27  Solve  logistical  problems  59 

28  Solve  tactical  maneuver  problems  60 

29  Solve  administrative  problems  60 

30  Solve  leadership  problems  60 

31  Solve  medical  problems  60 

32  Solve  communication  problems  60 

33  Operate  precision  hand-held  equipment  60 

34  Operate  hand-held  tools  60 

35  Operate  hand-held  power  equipment  60 

36  Operate  larger  power  equipment  60 

37  Operate  full  keyboard  60 

38  Operate  numeric  keyboard  60 

39  Adjust  control  device  using  one  limb  60 

40  Adj  control  device  using  multiple  lim  60 

41  Drive  tracked  vehicle  60 

42  Drive  heavy  wheeled  vehicle  60 

43  Drive  light  wheeled  vehicle  60 

44  Aim  stationary  target  59 

45  Aim  moving  target  59 

46  Walk  long  distances  59 

47  Run  short  distances  59 

48  Push,  pull,  lift  heavy  weights  59 

49  Throw  objects  59 

50  Sort,  fold,  feed  by  hand  59 

51  Make  coordinated  movements  59 

52  Work  long  hours  59 

53  Work  under  adverse  conditions  59 


portance  Frequency  Difficulty 

MEAN  S.D.  N  MEAN  S.D.  N  MEAN  S.O. 

4.68  0.57  60  2.87  0.39  60  1.77  0.65 

3.30  1.96  60  1.68  1.16  47  2.30  0.78 

2.65  1.86  60  1.35  1.01  44  1.84  0.71 

1.95  1.96  60  0.98  1.08  33  2.03  0.81 

3.25  1.76  60  1.68  1.03  50  1.84  0.77 

3.05  1.70  59  1.61  1.05  48  1.90  0.78 

3.28  1.53  60  1.85  0.95  53  1.75  0.62 

2.10  1.92  60  1.02  1.00  37  1.92  0.68 

3.60  1.52  60  1.95  0.93  54  1.67  0.70 

2.85  2.06  60  1.58  1.18  43  1.44  0.63 

2.92  1.81  60  1.58  1.05  47  1.68  0.73 

2.57  1.83  60  1.47  1.11  44  1.48  0.70 

0.28  0.94  60  0.13  0.47  6  2.00  0.63 

2.77  1.96  60  1.48  1.14  43  1.84  0.75 

3.68  1.50  60  2.03  0.94  55  1.69  0.72 

1.87  2.05  60  1.00  1.16  30  1.90  0.71 

2.62  2.04  60  1.33  1.08  40  1.77  0.73 

2.55  2.00  60  1.33  1.10  40  1.95  0.68 

2.72  1.90  60  1.42  1.08  43  1.98  0.67 

4.59  0.62  59  2.73  0.52  59  1.75  0.63 

4.15  1.05  60  2.42  0.74  58  1.50  0.60 

3.87  1.36  60  2.07  0.86  55  1.71  0.57 

4.00  1.30  60  2.07  0.84  56  2.16  0.63 

2.50  1.83  60  1.27  1.04  41  1.88  0.56 

1.37  1.86  60  0.65  0.92  24  2.33  0.56 

2.20  1.93  60  1.08  1.01  37  2.14  0.71 

0.47  1.24  59  0.25  0.66  9  2.22  0.67 

1.88  2.16  60  0.97  1.16  28  2.21  0.74 

0.52  1.27  60  0.37  0.84  11  2.27  0.65 

1.60  2.05  60  0.73  0.99  25  2.52  0.65 

0.73  1.66  60  0.37  0.82  11  2.27  0.65 

1.37  1.92  60  0.65  0.95  22  1.95  0.79 

0.52  1.37  60  0.25  0.68  9  2.11  0.78 

2.28  1.81  60  1.40  1.15  42  1.21  0.52 

0.72  1.47  60  0.40  0.83  14  1.43  0.51 

0.12  0.64  60  0.07  0.41  2  1.50  0.71 

0.42  1.06  60  0.22  0.58  8  2.00  0.53 

0.12  0.69  60  0.03  0.18  2  2.50  0.71 

2.10  1.98  60  1.27  1.21  34  1.38  0.60 

1.98  2.02  60  1.12  1.17  32  1.53  0.62 

3.45  1.64  60  1.93  1.02  52  1.62  0.69 

2.30  1.96  60  1.22  1.15  38  1.63  0.75 

3.25  1.59  60  1.83  0.98  54  1.39  0.56 

4.80  0.45  59  2.54  0.65  59  1.64  0.64 

4.69  0.77  59  2.47  0.73  58  2.10  0.69 

4.08  1.07  59  2.49  0.68  58  1.62  0.79 

4.36  0.71  59  2.73  0.52  59  1.46  0.68 

3.58  1.45  59  2.24  0.92  53  1.55  0.70 

3.81  1.22  59  2.05  0.82  56  1.36  0.59 

1.59  1.81  59  0.85  1.00  29  1.17  0.38 

3.39  1.61  59  1.80  0.98  52  1.54  0.61 

3.93  1.27  59  2.41  0.81  57  1.84  0.75 

4.15  1.03  59  2.41  0.75  58  1.86  0.74 


Table  4b 

Job  Activity  Importance,  Frequency,  and  Difficulty  Rating  Descriptive  Statistics 

63B  Vehicle/Generator  Mechanic 


No. Job  Activity  N 

1  Work  in  a  team  35 

2  Lead  a  team  35 

3  Support/counsel  peers  35 

4  Support/counsel  subordinates  35 

5  Coach  peers  34 

6  Coach  subordinates  34 

7  Make  oral  reports  (to  individuals)  35 

8  Make  oral  reports  (to  groups)  35 

9  Relay  oral  instructions  35 

10  Ask  questions  35 

11  Record  information  34 

12  Write  brief  messages  35 

13  Write  longer  reports  35 

14  Monitor/interpret  verbal  messages  35 

15  Recall  verbal  information  35 

16  Monitor/interpret  numerical  informati  34 

17  Recall  numerical  information  35 

18  Monitor/interpret  figural  information  35 

19  Recall  figural  information  34 

20  Follow  oral  directions  35 

21  Follow  written  directions  35 

22  Judge  size  and  distance  35 

23  Judge  location  35 

24  Judge  paths  of  moving  objects  35 

25  Solve  electrical  system  problems  35 

26  Solve  mechanical  system  problems  35 

27  Solve  logistical  problems  35 

28  Solve  tactical  maneuver  problems  35 

29  Solve  administrative  problems  35 

30  Solve  leadership  problems  35 

31  Solve  medical  problems  35 

32  Solve  communication  problems  35 

33  Operate  precision  hand-held  equipment  35 

34  Operate  hand-held  tools  35 

35  Operate  hand-held  power  equipment  35 

36  Operate  larger  power  equipment  35 

37  Operate  full  keyboard  35 

38  Operate  numeric  keyboard  35 

39  Adjust  control  device  using  one  limb  35 

40  Adj  control  device  using  multiple  lim  35 

41  Drive  tracked  vehicle  35 

42  Drive  heavy  wheeled  vehicle  35 

43  Drive  light  wheeled  vehicle  35 

44  Aim  stationary  target  35 

45  Aim  moving  target  35 

46  Walk  long  distances  35 

47  Run  short  distances  35 

48  Push,  pull,  lift  heavy  weights  35 

49  Throw  objects  35 

50  Sort,  fold,  feed  by  hand  35 

51  Make  coordinated  movements  35 

52  Work  long  hours  35 

53  Work  under  adverse  conditions  35 


iportance  Frequency  Difficulty 

MEAN  S.D.  N  MEAN  S.D.  N  MEAN  S.D. 

3.66  1.41  35  2.20  0.96  33  1.73  0.76 

1.77  1.93  35  0.86  1.03  18  2.11  0.83 

2.06  1.92  35  1.17  1.15  21  1.95  0.80 

1.69  1.91  35  0.91  1.07  18  2.28  0.75 

2.32  1.74  34  1.29  1.06  24  1.92  0.65 

2.41  1.88  34  1.24  1.05  24  2.00  0.72 

3.00  1.81  35  1.71  1.15  29  1.83  0.80 

0.54  0.95  35  0.37  0.60  11  2.09  0.83 

2.89  1.59  35  1.60  0.98  31  1.61  0.62 

2.57  2.20  35  1.57  1.31  23  1.35  0.57 

1.32  1.90  34  0.71  1.06  12  1.67  0.65 

1.09  1.60  35  0.66  0.97  13  1.69  0.75 

0.14  0.43  35  0.11  0.32  4  2.75  0.50 

1.86  1.87  35  1.03  1.07  20  1.75  0.72 

3.03  1.72  35  1.66  1.03  29  1.83  0.71 

1.29  1.77  34  0.76  1.02  14  1.79  0.70 

2.51  1.74  35  1.43  0.98  28  1.96  0.74 

3.11  1.78  35  1.71  1.02  31  1.97  0.60 

2.50  1.96  34  1.29  1.03  25  1.96  0.73 

4.49  0.92  35  2.63  0.60  35  1.77  0.69 

4.60  0.81  35  2.71  0.52  35  1.89  0.63 

2.71  1.72  35  1.57  1.09  28  1.61  0.79 

2.83  1.60  35  1.37  1.00  30  1.97  0.81 

2.83  1.52  35  1.60  0.98  32  1.69  0.82 

4.20  1.28  35  2.26  0.82  33  2.55  0.56 

4.34  1.30  35  2.43  0.78  33  2.36  0.65 

1.54  1.82  35  0.77  1.00  17  2.18  0.81 

0.46  1.22  35  0.14  0.36  5  2.40  0  89 

0.91  1.38  35  0.37  0.55  12  2.17  0.72 

0.69  1.25  35  0.29  0.52  9  2.00  0  71 

0.31  0.99  35  0.11  0.32  4  2.00  0.00 

0.77  1.50  35  0.37  0.73  9  2.44  0.73 

3.69  1.51  35  1.89  0.96  32  2.06  0.67 

4.60  0.88  35  2.74  0.56  35  1.46  0.70 

3.94  1.45  35  2.20  0.93  33  1.61  0.70, 

1.54  1.93  35  0.80  1.08  15  1.60  0.63 

0.29  0.75  35  0.17  0.45  5  1.40  0.55 

0.31  0.96  35  0.11  0.32  4  1.75  0.50 

3.00  1.88  35  1.74  1.09  28  1.54  0.74 

3.49  1.70  35  2.06  1.03  31  1.71  0.78 

1.74  1.90  35  0.94  1.11  18  1.83  0.51 

2.80  2.01  35  1.60  1.22  25  1.72  0.68 

4.34  0.94  35  2.54  0.61  35  1.57  0.70 

3.11  1.41  34  1.35  0.81  32  1.53  0.62 

2.14  1.72  34  0.88  0.77  24  1.63  0.65 

2.71  1.64  34  1.32  0.91  29  1.24  0.58 

2.69  1.66  34  1.82  1.11  30  1.33  0.61 

3.60  1.19  33  2.15  0.83  33  1.64  0.65 

1.46  1.74  35  0.83  0.98  18  1.11  0.47 

1.37  1.61  35  0.74  0.85  19  1.16  0.50 

3.31  1.53  35  1.77  1.00  32  1.50  0.62 

3.80  1.28  35  2.09  0.89  34  1.76  0  78 

3.80  1.32  35  2.06  0.87  34  1.94  0  69 


Table  4c 

Job  Activity  Importance,  Frequency,  and  Difficulty  Rating  Descriptive  Statistics 

71L  Administrative  Specialist 


No. Job  Activity  N 

1  Work  in  a  team  39 

2  Lead  a  team  39 

3  Support/counsel  peers  39 

4  Support/counsel  subordinates  39 

5  Coach  peers  39 

6  Coach  subordinates  39 

7  Make  oral  reports  (to  individuals)  39 

8  Make  oral  reports  (to  groups)  39 

9  Relay  oral  instructions  39 

10  Ask  questions  39 

11  Record  information  39 

12  Write  brief  messages  39 

13  Write  longer  reports  39 

14  Monitor/interpret  verbal  messages  39 

15  Recall  verbal  information  39 

16  Monitor/interpret  numerical  informati  39 

17  Recall  numerical  information  39 

18  Monitor/interpret  figural  information  39 

19  Recall  figural  information  39 

20  Follow  oral  directions  39 

*  21  Follow  written  directions  39 

22  Judge  size  and  distance  39 

23  Judge  location  . 39 

24  Judge  paths  of  moving  objects  39 

25  Solve  electrical  system  problems  39 

26  Solve  mechanical  system  problems  39 

27  Solve  logistical  problems  38 

28  Solve  tactical  maneuver  problems  39 

29  Solve  administrative  problems  39 

30  Solve  leadership  problems  39 

31  Solve  medical  problems  39 

32  Solve  communication  problems  39 

33  Operate  precision  hand-held  equipment  39 

34  Operate  hand-held  tools  39 

35  Operate  hand-held  power  equipment  39 

36  Operate  larger  power  equipment  39 

37  Operate  full  keyboard  39 

38  Operate  numeric  keyboard  39 

39  Adjust  control  device  using  one  limb  39 

40  Adj  control  device  using  multiple  lim  39 

41  Drive  tracked  vehicle  39 

42  Drive  heavy  wheeled  vehicle  39 

43  Drive  light  wheeled  vehicle  39 

44  Aim  stationary  target  39 

45  Aim  moving  target  39 

46  Walk  long  distances  39 

47  Run  short  distances  39 

48  Push,  pull,  lift  heavy  weights  39 

49  Throw  objects  39 

50  Sort,  fold,  feed  by  hand  39 

51  Make  coordinated  movements  39 

52  Work  long  hours  39 

53  Work  under  adverse  conditions  39 


portance  Frequency  Difficulty 

MEAN  S.D.  N  MEAN  S.D.  N  MEAN  S.D. 

3.72  1.45  39  2.10  1.02  36  1.53  0.74 

1.59  1.90  39  0.79  1.03  19  2.11  0.81 

2.64  1.58  39  1.23  0.81  31  1.84  0.82 

1.85  1.97  39  0.82  0.91  20  2.00  0.79 

3.36  1.39  39  1.56  0.68  36  1.97  0.70 

2.79  1.88  39  1.31  0.92  29  1.79  0.73 

3.00  1.49  39  1.72  1.00  34  1.68  0.64 

1.13  1.47  39  0.59  0.79  17  2.18  0.73 

3.28  1.52  39  1.85  0.96  34  1.74  0.62 

2.00  1.85  39  1.21  1.17  23  1.61  0.66 

4.03  1.04  39  2.62  0.63  38  1.55  0.69 

2.87  1.98  39  1.72  1.26  28  1.75  0.80 

0.72  1.38  39  0.36  0.74  9  1.89  0.78 

2.95  1.64  39  1.77  1.04  32  1.78  0.66 

2.92  1.72  39  1.74  1.07  31  1.71  0.59 

1.69  1.73  39  0.92  1.04  21  1.67  0.66 

1.95  1.81  39  1.13  1.13  23  1.83  0.72 

1.26  1.43  39  0.69  0.83  19  1.84  0.60 

1.13  1.28  39  0.64  0.78  19  1.89  0.74 

4.21  0.89  39  2.56  0.60  38  1.71  0.52 

4.36  0.58  39  2.62  0.54  38  1.76  0.63 

0.97  1.31  39  0.49  0.64  16  1.69  0.70 

1.62  1.57  39  0.72  0.60  25  1.76  0.78 

0.51  0.97  39  0.26  0.44  10  1.70  0.82 

0.49  1.07  39  0.23  0.48  8  2.25  0.71 

0.54  1.05  39  0.26  0.44  10  2.20  0.63 

0.50  1.03  38  0.26  0.55  8  1.63  0.52 

0.21  0.66  39  0.13  0.41  4  2.00  0.82 

3.13  1.54  39  1.85  0.96  34  1.97  0.72 

1.36  1.66  39  0.69  0.86  18  2.17  0.79 

0.10  0.45  39  0.05  0.22  2  2.00  1.41 

0.95  1.45  39  0.51  0.82  13  1.85  0.80 

0.26  0.91  39  0.13  0.52  3  2.33  0.58 

0.33  0.81  39  0.26  0.59  8  1.38  0  74 

0.13  0.66  39  0.10  0.50  2  2.00  0.00 

0.13  0.66  39  0.10  0.50  2  2.50  0.71 

4.69  0.57  39  2.95  0.22  39  1.87  0.57 

3.00  2.08  39  1.79  1.32  28  1.79  0.69 

0.72  1.23  39  0.49  0.82  12  1.25  0.45 

0.41  0.94  39  0.28  0.65  7  1.29  0  49 

0.33  1.11  39  0.18  0.60  4  2.25  0.50 

0.64  1.39  39  0.31  0.69  8  1.88  0.64 

2.13  1.51  39  1.15  0.78  32  1.34  0.48 

2.13  1.87  39  0.92  0.84  26  1.65  0.56 

1.13  1.79  39  0.46  0.68  14  ’2.14  0.66 

1.41  1.60  39  0.69  0.73  22  1.55  0.67 

2.10  1.62  39  1.33  1.03  29  1.28  0.45 

1.10  1.31  39  0.64  0.71  20  1.40  0.50 

0.92  1.38  39  0.49  0.60  17  1.18  0.39 

2.72  1.34  39  1.64  0.90  35  1.11  0.32 

1.54  1.50  39  0.92  0.90  23  1.30  0.47 

2.49  1.62  39  1.23  0.90  30  1.43  0.57 

1.92  1.74  39  0.79  0.80  24  1.67  0.70 


Table  5a 

Job  Activity  Importance  Rating  Descript 
Sorted  by  Mean  Rating 
For  IIB:  Infantryman 


No. Job  Activity  N 

44  Aim  stationary  target  59 

45  Aim  moving  target  59 

1  Work  in  a  team  60 

20  Follow  oral  directions  59 

47  Run  short  distances  59 

53  Work  under  adverse  conditions  59 

21  Follow  written  directions  60 

46  Walk  long  distances  59 

23  Judge  location  60 

52  Work  long  hours  59 

22  Judge  size  and  distance  60 

49  Throw  objects  59 

15  Recall  verbal  information  60 

9  Relay  oral  instructions  60 

48  Push,  pull,  lift  heavy  weights  59 

41  Drive  tracked  vehicle  60 

51  Make  coordinated  movements  59 

2  Lead  a  team  60 

7  Make  oral  reports  (to  individuals)  60 

5  Coach  peers  60 

43  Drive  light  wheeled  vehicle  60 

6  Coach  subordinates  59 

11  Record  information  60 

10  Ask  questions  60 

14  Monitor/ interpret  verbal  messages  60 

19  Recall  figural  information  60 

3  Support/counsel  peers  60 

17  Recall  numerical  information  60 

12  Write  brief  messages  60 

18  Monitor/interpret  figural  information  60 

24  Judge  paths  of  moving  objects  60 

42  Drive  heavy  wheeled  vehicle  60 

34  Operate  hand-held  tools  60 

26  Solve  mechanical  system  problems  60 

8  Make  oral  reports  (to  groups)  60 

39  Adjust  control  device  using  one  limb  60 

40  Adj  control  device  using  multiple  lim  60 

4  Support/counsel  subordinates  60 

28  Solve  tactical  maneuver  problems  60 

16  Monitor/interpret  numerical  informati  60 

30  Solve  leadership  problems  60 

50  Sort,  fold,  feed  by  hand  59 

25  Solve  electrical  system  problems  60 

32  Solve  conmiuni cation  problems  60 

31  Solve  medical  problems  60 

35  Operate  hand-held  power  equipment  60 

29  Solve  administrative  problems  60 

33  Operate  precision  hand-held  equipment  60 

27  Solve  logistical  problems  59 

37  Operate  full  keyboard  60 

13  Write  longer  reports  60 

36  Operate  larger  power  equipment  60 

38  Operate  numeric  keyboard  60 


ive  Statistics 


MEAN  S.D. 

4.80  0.45 
4.69  0.77 

4.68  0.57 

4.59  0.62 

4.36  0.71 
4.15  1.03 
4.15  1.05 
4.08  1.07 
4.00  1.30 
3.93  1.27 

3.87  1.36 

3.81  1.22 

3.68  1.50 

3.60  1.52 

3.58  1.45 
3.45  1.64 
3.39  1.61 

3.30  1.96 

3.28  1.53 

3.25  1.76 
3.25  1.59 

3.05  1.70 
2.92  1.81 

2.85  2.06 
2.77  1.96 
2.72  1.90 

2.65  1.86 
2.62  2.04 
2.57  1.83 
2.55  2.00 
2.50  1.83 

2.30  1.96 

2.28  1.81 

2.20  1.93 
2.10  1.92 
2.10  1.98 
1.98  2.02 
1.95  1.96 

1.88  2.16 
1.87  2.05 

1.60  2.05 

1.59  1.81 

1.37  1.86 
1.37  1.92 
0.73  1.66 
0.72  1.47 
0.52  1.27 
0.52  1.37 
0.47  1.24 
0.42  1.06 
0.28  0.94 
0.12  0.64 
0.12  0.69 


Table  5b 

Job  Activity  Importance  Rating  Descriptive  Statistics 
Sorted  by  Mean  Rating 
For  63B;  Vehicle/  Generator  Mechanic 
No. Job  Activity  N  MEAN  S.D. 

21  Follow  written  directions  35  4.60  0.81 

34  Operate  hand-held  tools  35  4.60  0.88 

20  Follow  oral  directions  35  4.49  o.92 

26  Solve  mechanical  system  problems  35  4.34  1.30 

43  Drive  light  wheeled  vehicle  35  4.34  0.94 

25  Solve  electrical  system  problems  35  4.20  1.28 

35  Operate  hand-held  power  equipment  35  3.94  1.45 

52  Work  long  hours  35  3. 80  1.28 

53  Work  under  adverse  conditions  35  3.80  1.32 

33  Operate  precision  hand-held  equipment  35  3.69  1.51 

1  Work  in  a  team  35  3.66  1.41 

48  Push,  pull,  lift  heavyweights  35  3.60  1.19 

40  Adj  control  device  using  multiple  lim  35  3.49  1.70 

51  Make  coordinated  movements  35  3.31  1.53 

18  Monitor/interpret  figural  information  35  3.11  1.78 

44  Aim  stationary  target  35  3.11  1.41 

15  Recall  verbal  information  35  3.03  1.72 

7  Make  oral  reports  (to  individuals)  35  3.00  1.81 

39  Adjust  control  device  using  one  limb  35  3.00  1.88 

9  Relay  oral  instructions  35  2.89  1.59 

23  Judge  location  35  2.83  1.60 

24  Judge  paths  of  moving  objects  35  2.83  1.52 

42  Drive  heavy  wheeled  vehicle  35  2.80  2.01 

22  Judge  size  and  distance  35  2.71  1.72 

46  Walk  long  distances  35  2.71  1.64 

47  Run  short  distances  35  2.69  1  66 

}5  Jsk  questions  35  2.57  2!20 

17  Recall  numerical  information  35  2.51  1.74 

19  Recall  figural  information  34  2.50  1.96 

6  Coach  subordinates  34  2  41  1  88 

5  Coach  peers  34  2.’32  1.74 

45  Aim  moving  target  35  2.14  l!72 

3  Support/counsel  peers  35  2. 06  1.92 

14  Monitor/ interpret  verbal  messages  35  1.86  1  87 

2  Lead  a  team  35  1.77  1.93 

41  Drive  tracked  vehicle  35  1.74  1.90 

4  Support/counsel  subordinates  35  1.69  1.91 

27  Solve  logistical  problems  35  1.54  1.82 

36  Operate  larger  power  equipment  35  1.54  1.93 

49  Throw  objects  35  1.46  1.74 

50  Sort,  fold,  feed  by  hand  35  1.37  1.61 

11  Record  information  34  1.32  1.90 

16  Monitor/ interpret  numerical  informati  34  l!29  l!77 

12  Write  brief  messages  35  i.09  1.60 

29  Solve  administrative  problems  35  0.91  l!38 

32  Solve  communication  problems  35  0.77  1.50 

30  Solve  leadership  problems  35  0.69  1.25 

8  Make  oral  reports  (to  groups)  35  0.54  0.95 

28  Solve  tactical  maneuver  problems  35  0.46  1.22 

31  Solve  medical  problems  35  0.31  0.99 

38  Operate  numeric  keyboard  35'  0.31  o!96 

37  Operate  full  keyboard  35  0.29  0.75 

13  Write  longer  reports  35  0.14  0.43 


Table  5c 

Job  Activity  Importance  Rating  Descriptive  Statisti 
Sorted  by  Mean  Rating 
For  71L:  Administrative  Specialist 
No. Job  Activity  N  MEAN  S.D. 

37  Operate  full  keyboard  39  4.69  0.57 

21  Follow  written  directions  39  4.36  0.58 

20  Follow  oral  directions  39  4.21  0.89 

11  Record  information  39  4.03  1.04 

1  Work  in  a  team  39  3.72  1.45 

5  Coach  peers  39  3.36  1.39 

9  Relay  oral  instructions  39  3.28  1.52 

29  Solve  administrative  problems  39  3.13  1.54 

7  Make  oral  reports  (to  individuals)  39  3.00  1.49 

38  Operate  numeric  keyboard  39  3.00  2.08 

14  Monitor/ interpret  verbal  messages  39  2.95  1.64 

15  Recall  verbal  information  39  2.92  1.72 

12  Write  brief  messages  39  2.87  1.98 

6  Coach  subordinates  39  2.79  1.88 

50  Sort,  fold,  feed  by  hand  39  2.72  1.34 

3  Support/counsel  peers  39  2.64  1.58 

52  Work  long  hours  39  2.49  1.62 

43  Drive  light  wheeled  vehicle  39  2.13  l!51 

44  Aim  stationary  target  39  2.13  1.87 

47  Run  short  distances  39  2.10  1.62 

10  Ask  questions  39  2.00  1.85 

17  Recall  numerical  information  39  1.95  1.81 

53  Work  under  adverse  conditions  39  1.92  1.74 

4  Support/counsel  subordinates  39  1.85  1.97 

16  Monitor/ interpret  numerical  informati  39  1.69  1.73 

23  Judge  location  39  i.62  1.57 

2  Lead  a  team  39  1.59  1.90 

51  Make  coordinated  movements  39  1.54  1.50 

46  Walk  long  distances  39  1.41  1.60 

30  Solve  leadership  problems  39  1.36  1.66 

18  Monitor/ interpret  figural  information  39  1.26  1.43 

8  Make  oral  reports  (to  groups)  39  1.13  1.47 

19  Recall  figural  information  39  1.13  1.28 

45  Aim  moving  target  39  1.13  1.79 

48  Push,  pull,  lift  heavyweights  39  1.10  L31 

22  Judge  size  and  distance  39  o.97  1.31 

32  Solve  communication  problems  39  0.95  l!45 

49  Throw  objects  39  o.92  1.38 

13  Write  longer  reports  39  0.72  1.38 

39  Adjust  control  device  using  one  limb  39  0.72  1.23 

42  Drive  heavy  wheeled  vehicle  39  0.64  1.39 

26  Solve  mechanical  system  problems  39  0.54  1.05 

24  Judge  paths  of  moving  objects  39  0.51  0.97 

27  Solve  logistical  problems  38  0.50  1.03 

25  Solve  electrical  system  problems  39  0.49  1.07 

40  Adj  control  device  using  multiple  lim  39  0.41  0.94 

34  Operate  hand-held  tools  39  0.33  0.81 

41  Drive  tracked  vehicle  39  0.33  1.11 

33  Operate  precision  hand-held  equipment  39  0.26  0.91 

28  Solve  tactical  maneuver  problems  39  0.21  0.66 

35  Operate  hand-held  power  equipment  39  0.13  0.66 

36  Operate  larger  power  equipment  39  0.13  0.66 

31  Solve  medical  problems  39  0.10  0.45 


Table  6a 

Task  Category  Mean  Within-SME  Correlations* 
between  Importance,  Frequency,  and  Difficulty  Ratings 


Importance 

Frequency 

IIB  Difficulty 

.11 

-.09 

(n=60)  Frequency 

.48 

63B  Difficulty 

.25 

.13 

(n=35)  Frequency 

.57 

71L  Difficulty 

.22 

.14 

(n=39)  Frequency 

.63 

*  Ratings  set  to  missing  when  not  part  of  job 


Table  6b 

Job  Activity  Mean  Within-SME  Correlations* 
between  Importance,  Frequency,  and  Difficulty  Ratings 


IIB  Difficulty 
(n=60)  Frequency 

63B  Difficulty 
(n=35)  Frequency 

71L  Difficulty 
(n=39)  Frequency 


Importance  Frequency 


.08 

-.02 

.56 

.21 

.14 

.70 

.20 

.07 

.70 

*  Ratings  set  to  missing  when  not  part  of  job 


Table  7 


Variance  Component  and  Reliability  Estimates 
For  Task  Category  and  Activity  Importance  Ratings 

Across  3  MOS 

VARIANCE  COMPONENTS 


Task  Activity 


Components 

Variance 

Percent  Variance 

Percent 

Component 

1.059 

30.51% 

0.495 

13.48% 

MOS 

0.025 

0.73% 

0.111 

3.02% 

Component  x  MOS 

0.921 

26.54% 

1.062 

28.91% 

Rank 

0.000 

0.00% 

0.000 

0.00% 

Command 

0.000 

0.00% 

0.002 

0.06% 

Rank  x  Command 

0.041 

1.19% 

0.000 

0.00% 

Rank  x  MOS 

0.036 

1.04% 

0.108 

2.94% 

Command  x  MOS 

0.045 

1.30% 

0.032 

0.88% 

Rk.  X  C  X  M 

0.003 

0.09% 

0.000 

0.00% 

Component  x  Rank 

0.038 

1.09% 

0.051 

1.39% 

Component  x  Command 

0.039 

1.12% 

0.035 

0.95% 

Comp  X  Rank  x  Command 

0.000 

0.00% 

0.000 

0.00% 

Comp  X  Rank  x  MOS 

0.027 

0.78% 

0.033 

0.91% 

Comp  X  Command  x  MOS 

0.068 

1.96% 

0.064 

1.74% 

Rank  x  Command  x  MOS 

0.003 

0.09% 

0.000 

0.00% 

Cp  X  Rk  X  Cd  X  M 

0.101 

2.91% 

0.000 

0.00% 

Rater(Rk  Cd  M) 

0.096 

2.77% 

0.184 

5.02% 

Component  x  Rater 

0.968 

27.88% 

1.495 

40.71% 

Total 

3.470 

100.00% 

3.673 

100.00% 

RELIABILITY 


Overall 
Within  Rank 
Within  Command 
Within  Rank  &  Command 


0.58 

0.45 

0.62 

0.48 

0.63 

0.47 

0.65 

0.50 

Table  8a 


Variance  Component  and  Reliability  Estimates 
For  Task  Category  and  Activity  Importance  Ratings 
NCOS  Across  3  MOS 

VARIANCE  COMPONENTS 


Task  Category 

Job  Activity 

Variance 

Percent 

Variance 

Percent 

Component 

1.03 

29.89% 

0.57 

15.14% 

MOS 

0.07 

2.13% 

0.24 

6.23% 

Component*MOS 

1.01 

29.22% 

1.05 

27.69% 

Command 

0.04 

1.27% 

0.00 

0.00% 

Component*Command 

0.00 

0.00% 

0.00 

0.00% 

MOS*Command 

0.06 

1.86% 

0.02 

0.53% 

Rater(Cd  M) 

0.11 

3.29% 

0.18 

4.80% 

Component*Rater(Cd  M) 

1.11 

32.33% 

1.73 

45.60% 

3.44 

100.00% 

3.79 

100.00% 

RELIABILITY 

Overal 1 

0.61 

0.49 

Within  Command 

0.63 

0.49 

Table  8b 

Variance  Component  and  Reliability  Estimates 
For  Task  Category  and  Activity  Importance  Ratings 
Officers  Across  3  MOS 


Component 

MOS 

Component*MOS 

Command 

Component*Command 
MOS*Command 
Rater{Cd  M) 
Component*Rater(Cd  M) 


VARIANCE  COMPONENTS 


Task  Category  Job  Activity 


Variance 

Percent 

Variance 

Percent 

1.31 

39.16% 

0.46 

14.15% 

0.04 

1.12% 

0.17 

5.30% 

0.77 

22.95% 

1.24 

38.19% 

0.00 

0.00% 

0.00 

0.00% 

0.00 

0.00% 

0.05 

1.66% 

0.02 

0.54% 

0.00 

0.00% 

0.13 

3.74% 

0.07 

2.27% 

1.09 

32.50% 

1.25 

38.43% 

3.35  100.00% 


3.24  100.00% 


RELIABILITY 

0.63 

0.64 


0.58 

0.59 


Overal 1 

Within  Command 


Variance  Component  and  Reliability  Estimates 
For  Task  Category  and  Activity  Importance  Ratings 
FORSCOM  SMEs  Across  3  MOS 


VARIANCE  COMPONENTS 

Task  Category  Job  Activity 


Variance 

Percent 

Variance 

Component 

1.05 

30.00% 

0.50 

14.53% 

MOS 

0.14 

3.91% 

0.27 

7.82% 

Component*MOS 

1.09 

31.28% 

1.04 

30.04% 

Rank 

0.00 

0.00% 

0.00 

0.00% 

Component*Rank 

0.00 

0.00% 

0.01 

0.27% 

M0S*Rank 

0.05 

1.33% 

0.05 

1.42% 

Rater(Rk  M) 

0.11 

3.28% 

0.13 

3.77% 

Component*Rater(Rk  M) 

1.06 

30.20% 

1.46 

42.16% 

3.50 

100.00% 

3.45 

100.00% 

RELIABILITY 

Overal 1 

0.65 

0.52 

Within  Command 

0.66 

0.53 

Table  9b 

Variance  Component  and  Reliability  Estimates 
For  Task  Category  and  Activity  Importance  Ratings 
DOTD  SMEs  Across  3  MOS 

VARIANCE  COMPONENTS 


Task  Category 

Job  Activity 

Component 

Variance 

Percent 

Variance 

Percent 

1.14 

35.30% 

0.55 

14.26% 

MOS 

.00 

0.07% 

0.03 

0.88% 

Component*MOS 

0.89 

27.76% 

1.22 

31.40% 

Rank 

0.02 

0.58% 

0.00 

0.00% 

Component*Rank 

0.00 

0.00% 

0.00 

0.00% 

M0S*Rank 

.00 

0.05% 

0.09 

2.30% 

Rater(Rk  M) 

0.15 

4.65% 

0.35 

8.91% 

Component*Rater(Rk  M) 

1.02 

31.59% 

1.64 

42.25% 

3.22 

100.00% 

3.88 

100.00% 

RELIABILITY 

Overall  0.63  0.47 

Within  Command  0.64  0.48 


Table  10a 


Variance  Component  and  Reliability  Estimates 
For  Task  Category  Importance  Ratings 
Within  each  MOS 

VARIANCE  COMPONENTS 


IIB  63B  71L 


Components 

Variance 

Percent  Variance 

Percent  Variance 

Percent 

Task 

2.21 

50.76% 

2.11 

58.59% 

1.48 

50.60% 

Rank 

0.01 

0.32% 

0.01 

0.39% 

0.02 

0.57% 

Command 

0.08 

1.82% 

0.01 

0.37% 

.00 

0.10% 

Rank  x  Command 

0.08 

1.86% 

.00 

0.04% 

0.01 

0.50% 

Rater 

0.12 

2.76% 

0.06 

1.68% 

0.16 

5.38% 

Task  X  Rater 

1.67 

38.33% 

1.01 

27.92% 

1.21 

41.44% 

Task  X  Rank 

0.10 

2.37% 

0.05 

1.43% 

0.01 

0.47% 

Task  X  Command 

0.08 

1.78% 

0.24 

6.68% 

0.03 

0.94% 

Task  X  Rank  x  Command 

0.00 

0.00% 

0.10 

2.90% 

0.00 

0.00% 

Total 

4.35 

100.00% 

3.60 

100.00% 

2.93 

100.00% 

RELIABILITY 

IIB 

63B 

71L 

Overall 

0.51 

0.59 

0.51 

Within  Rank 

0.52 

0.61 

0.51 

Within  Command 

0.53 

0.65 

0.51 

Within  Rank  &  Command 

0.55 

0.66 

0.52 

Table  10b 


Variance  Component  and  Reliability  Estimates 
For  Activity  Importance  Ratings 
Within  each  MOS 


VARIANCE  COMPONENTS 


IIB  63B  71L 


Components 

Variance 

Percent  Variance 

Percent  Variance 

Percent 

Activity 

1.72 

40.31% 

1.40 

36.47% 

1.51 

42.73% 

Rank 

0.03 

0.62% 

0.03 

0.66% 

0.03 

0.91% 

Command 

0.03 

0.74% 

.00 

0.06% 

0.01 

0.30% 

Rank  x  Command 

0.01 

0.25% 

0.01 

0.30% 

0.00 

0.00% 

Rater 

0.35 

8.16% 

0.21 

5.44% 

0.23 

6.62% 

Activity  x  Rater 

1.97 

46.18% 

1.56 

40.59% 

1.70 

47.82% 

Activity  x  Rank 

0.11 

2.67% 

0.20 

5.14% 

0.00 

0.00% 

Activity  x  Command 

0.05 

1.07% 

0.44 

11.33% 

0.00 

0.00% 

Act.  X  Rank  x  Command 

0.00 

0.00% 

0.00 

0.00% 

0.06 

1.61% 

Total 

4.27 

100.00% 

3.85 

100.00% 

3.55 

100.00% 

RELIABILITY 


IIB 

63  B 

71L 

Overall 

0.40 

0.36 

0.43 

Within  Rank 

0.42 

0.39 

0.44 

Within  Command 

0.41 

0.41 

0.44 

Within  Rank  &  Command 

0.43 

0.44 

0.44 

Table  11a 


Variance  Component  and  Reliability  Estimates 
For  Task  Category  Importance  Ratings 
For  NCOS  within  each  MOS 

VARIANCE  COMPONENTS 

IIB  63B  71L 

Variance  Percent  Variance  Percent  Variance  Percent 
Task  2.30  48.93%  2.08  57.15%  1.32  47.28% 

Command  0.14  2.90%  0.01  0.37%  0.00  0.00% 

Rater  (Command)  0.19  4.06%  0.10  2.88%  0.19  6.69% 

Task*Rater  1.98  41.95%  0.99  27.19%  1.29  46.03% 

Task*Command  0.10  2.16%  0.45  12.41%  0.00  0.00% 


4.71  100.00%  3.64  100.00%  2.80  100.00% 


RELIABILITY 

0.49  0.57  0.47 

0.52  0.66  0.47 


Overal 1 

Within  Command 


Table  11b 

Variance  Component  and  Reliability  Estimates 
For  Task  Category  Importance  Ratings 
For  Officers  within  each  MOS 

VARIANCE  COMPONENTS 


Task 
Command 
Rater (Command) 
Task*Rater 
Task*Command 


IIB  63B  71L 

Variance  Percent  Variance  Percent  Variance  Percent 
2.41  57.20%  2.28  62.94%  1.76  55.74% 

.00  0.02%  .00  0.03%  0.03  0.94% 

0.20  4.83%  0.04  1.00%  0.16  5.14% 

1.60  37.95%  1.30  36.02%  1.20  38.19% 

0.00  0.00%  0.00  0.00%  0.00  0.00% 


4.21  100.00%  3.62  100.00%  3.15  100.00% 


RELIABILITY 


Overal 1 

Within  Command 


0.57 

0.57 


0.63 

0.63 


0.56 

0.56 


Table  11c 


Variance  Component  and  Reliability  Estimates 
For  Job  Activity  Importance  Ratings 
For  NCOS  within  each  MOS 


VARIANCE  COMPONENTS 


Act 

Command 
Rater (Command) 
Act*Rater 
Act*Command 


IIB  63B  71L 


Variance 

Percent  Variance 

1.76 

37.90% 

1.61 

0.03 

0.56% 

0.00 

0.44 

9.54% 

0.30 

2.41 

52.00% 

1.82 

0.00 

0.00% 

0.39 

Percent  Variance 

Percent 

39.01% 

1.33 

38.77% 

0.00% 

0.03 

0.81% 

7.26% 

0.26 

7.63% 

44.24% 

1.81 

52.80% 

9.48% 

0.00 

0.00% 

4.63  100.00%  4.12  100.00%  3.43  100.00% 


RELIABILITY 


Overall  0.38  0.39 

Within  Command  0.38  0.43 


0.39 

0.39 


Table  lid 


Variance  Component  and  Reliability  Estimates 
For  Job  Activity  Importance  Ratings 
For  Officers  within  each  MOS 


VARIANCE  COMPONENTS 


Act 

Command 
Rater (Command) 
Act*Rater 
Act*Command 


IIB  63B 

Variance  Percent  Variance  Percent  Variance 


1.85 

48.87% 

1.39 

40.63% 

1.67 

0.01 

0.14% 

0.00 

0.00% 

0.01 

0.23 

6.01% 

0.08 

2.35% 

0.26 

1.70 

44.99% 

1.84 

53.85% 

1.64 

0.00 

0.00% 

0.11 

3.16% 

0.08 

3.79 

100.00% 

3.41 

100.00% 

3.65 

71L 

Percent 

45.71% 

0.15% 

7.05% 

45.03% 

2.06% 


100.00% 


RELIABILITY 

Overall  0.49  0.41  0.46 

Within  Command  0.49  0.42  0.47 


Table  12a 


Task 

Rank 

Rater(Rank) 

Task*Rater 

Task*Rank 


Overall 
Within  Rank 


Task 

Rank 

Rater(Rank) 

Task*Rater 

Task*Rank 


Overal 1 
Within  Rank 


Variance  Component  and  Reliability  Estimates 
For  Task  Category  Importance  Ratings 
For  FORSCOM  SMEs  within  each  MOS 

VARIANCE  COMPONENTS 


IIB  63B  71L 


Variance 

Percent  Variance 

Percent  Variance 

Percent 

2.38 

50.50% 

1.91 

55.64% 

1.55 

55.99% 

0.03 

0.69% 

0.00 

0.00% 

.00 

0.16% 

0.25 

5.33% 

0.11 

3.33% 

0.16 

5.75% 

1.98 

42.04% 

1.40 

41.03% 

1.05 

38.10% 

0.07 

1.44% 

0.00 

0.00% 

0.00 

0.00% 

4.72 

100.00% 

3.42 

100.00% 

2.76 

100.00% 

RELIABILITY 


0.51 

0.56 

0.56 

0.52 

0.56 

0.56 

Table  12b 


Variance  Component  and  Reliability  Estimates 
For  Task  Category  Importance  Ratings 
For  DOTD  SMEs  within  each  MOS 

VARIANCE  COMPONENTS 

IIB  63B  71L 

Variance  Percent  Variance  Percent  Variance  Percent 


2.13 

54.79% 

2.45 

64.36% 

1.51 

48.41% 

0.02 

0.46% 

0.02 

0.39% 

0.04 

1.43% 

0.21 

5.34% 

0.05 

1.33% 

0.18 

5.70% 

1.53 

39.41% 

1.22 

32.08% 

1.39 

44.46% 

0.00 

0.00% 

0.07 

1.83% 

0.00 

0.00% 

3.89 

100.00% 

3.80 

100.00% 

3.13 

100.00% 

RELIABILITY 

0.55 

0.64 

0.48 

0.55 

0.66 

0.49 

Table  12c 


Act 

Rank 

Rater{Rank) 

Act*Rater 

Act*Rank 


Overal 1 
Within  Rank 


Act 

Rank 

Rater (Rank) 

Act*Rater 

Act*Rank 


Overall 
Within  Rank 


Variance  Component  and  Reliability  Estimates 
For  Job  Activity  Importance  Ratings 
For  FORSCOM  SMEs  within  each  MOS 

VARIANCE  COMPONENTS 


IIB  63B  71L 


Variance 

Percent  Vari 

iance 

Percent  Variance 

Percent 

1.66 

39.55% 

1.20 

34.98% 

1.46 

43.82% 

0.01 

0.25% 

.00 

0.01% 

0.05 

1.60% 

0.34 

8.02% 

0.41 

12.01% 

0.16 

4.89% 

2.10 

50.02% 

1.82 

52.75% 

1.56 

46.89% 

0.09 

2.17% 

0.01 

0.26% 

0.09 

2.80% 

4.20 

100.00% 

3.44 

100.00% 

3.33 

100.00% 

RELIABILITY 

0.40 

0.35 

0.44 

0.41 

0.35 

0.46 

Table  12d 

Variance  Component  and  Reliability  Estimates 
For  Job  Activity  Importance  Ratings 
For  DOTD  SMEs  within  each  MOS 

VARIANCE  COMPONENTS 

IIB  63B  71L 

Variance  Percent  Variance  Percent  Variance  Percent 


1.81 

41.74% 

1.97 

45.43% 

1.52 

41.04% 

0.06 

1.42% 

0.12 

2.70% 

0.03 

0.69% 

0.43 

9.92% 

0.03 

0.73% 

0.31 

8.33% 

2.03 

46.92% 

2.13 

49.14% 

1.84 

49.94% 

0.00 

0.00% 

0.09 

2.00% 

0.00 

0.00% 

4.33 

100.00% 

4.34 

100.00% 

3.69 

100.00% 

RELIABILITY 

0.42 

0.45 

0.41 

0.42 

0.48 

0.41 

Table  13a 


Task  Category  and  Job  Activity 
Importance  Ratings  Single-SME  Reliability  Estimates*^ 

Across  3  MOS 


Task  Category 

Job  Activity 

Reliability 

No.  SMEs 
required 
for  rxx=90 

No.  SMEs 
Reliability  required 
for  rxx=' 

Overal 1 

58 

7 

45 

11 

NCO’s 

61 

6 

49 

9 

Officers 

63 

5 

58 

7 

FORSCOM 

65 

5 

52 

8 

DOTD 

63 

5 

47 

10 

*  Decimal  point  omitted 


Table  13b 


Task  Category 

Importance  Ratings  Single-SME  Reliability  Estimates^ 
Within  each  MOS 


IIB  63B  71L 


Pilot 
»'xx  N 

1 

Test 

req’d 

rxx=90 

Pre-Test 
rxx  N  req’d 
rxx=90 

Pilot  Test 
rxx  N  req’d 
rxx=90 

Pre-Test 
rxx  N  req’d 
rxx=90 

Pilot 
'^xx  N 

1 

Test 

req’d 

rxx=90 

Pre-Test 
rxx  N 

rxx=90 

Overall 

51 

9 

63  5 

59  6 

53  8 

51 

9 

44 

11 

NCO’s 

49 

9 

64  5 

57  7 

07  120 

47 

10 

37 

15 

Officers 

57 

7 

63  5 

63  5 

56  7 

56 

7 

34 

17 

FORSCOM 

51 

9 

56  7 

56 

7 

DOTD 

55 

7 

64  5 

48 

10 

Table  13c 

Job  Activitiy 

Importance  Ratings  Single-SME  Reliability  Estimates*' 

Within  each  MOS 


IIB 

63B 

71L 

Pilot  Test 
rxx  N  req’d 
rxx=90 

Pre-Test 

Pilot  Test 

Pre-Test 

Pilot  Test 

Pre-Test 

rxx  N  req’d 
rxx=90 

rxx  N  req’d 
rxx=90 

rxx  N  req’d 
rxx=90 

rxx  N  req’d 

rxx  N  req’d 
rxx=90 

Overall 

40 

14 

46 

11 

36 

16 

41 

13 

43 

12 

35 

17 

NCOS 

38 

15 

54 

8 

39 

14 

36 

16 

39 

14 

37 

15 

Officers 

49 

9 

40 

14 

41 

13 

56 

7 

46 

11 

34 

17 

FORSCOM 

40 

14 

35 

17 

44 

11 

DOTD 

42 

12 

45 

11 

41 

13 

*  Decimal  point  omitted 


Table  14a:  Tasks  Matched  to  Job  Component  with  50%  or  Greater  Agreement 
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26:  Put  on  field  or  pressure  dressing  88  75% 

27:  Navigate  from  point  to  point  81  65%  52  58% 

28:  Establish  observation  post 

29:  Conduct  day  &  night  surveillance 

30:  Administer  nerve  agent  antidote  75  58% 


Table  14b:  Tasks  Matched  to  Job  Component  with  50%  or  Greater  Agreement 
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Table  14c:  Tasks  Matched  to  Job  Component  with  50%  or  Greater  Agreement 
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Table  15a:  Summary  Table  for  Task  Category  Matching  Data 
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Table  15b:  Summary  Table  for  Job  Activity  Matching  Data 
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Table  16  Task  Category  Questionnaire  Evaluation 
Task  Category  Definitions 

1.  How  clear  were  the  task  category  definitions? 
(l=not  at  all  clear,  7=very  clear) 


N 

Mean 

S.D. 

Overall 

130 

6.03 

0.92 

IIB 

56 

6.00 

0.87 

63B 

35 

6.11 

0.87 

71L 

39 

6.00 

1.05 

NCOS 

75 

5.99 

1.03 

Officers 

45 

6.11 

0.65 

Civilians 

10 

6.00 

1.15 

FORSCOM 

63 

6.03 

0.88 

DOTD 

67 

6.03 

0.97 

2.  How  well  did  you  understand  the  terms  used  in  the  task  category 
definitions? 

(l=there  were  many  terms  I  did  not  understand,  7=there  were  no  terms  I 
did  not  understand) 


N 

Mean 

S.D. 

Overall 

130 

6.42 

0.74 

IIB 

56 

6.43 

0.68 

63B 

35 

6.34 

0.94 

71L 

39 

6.46 

0.64 

NCOS 

75 

6.44 

0.76 

Officers 

45 

6.42 

0.58 

Civilians 

10 

6.20 

1.23 

FORSCOM 

63 

6.51 

0.62 

DOTD 

67 

6.33 

0.84 

3.  What  percentage  of  your  job  was  covered  by  these  task  categories? 


N 

Mean 

S.D. 

Overall 

127 

81.10 

26.55 

IIB 

55 

76.73 

28.42 

63B 

34 

91.18 

15.72 

71L 

38 

78.42 

29.55 

NCOS 

74 

74.86 

30.03 

Officers 

44 

88.86 

18.70 

Civilians 

9 

94.44 

10.14 

FORSCOM 

62 

78.55 

29.30 

DOTD 

65 

83.54 

23.61 

Table  16  continued 


Task  Category  Ratings 

1.  How  clear  were  the  instructions  for  making  the  ratings? 
(l=not  at  all  clear,  7=very  clear) 


N 

Mean 

S.D. 

Overall 

123 

6.24 

1.09 

IIB 

54 

6.22 

1.18 

63  B 

33 

6.55 

0.67 

71L 

36 

5.97 

1.21 

NCOS 

72 

6.22 

1.13 

Officers 

41 

6.37 

0.70 

Civilians 

10 

5.80 

1.87 

FORSCOM 

58 

6.34 

0.89 

DOTD 

65 

6.14 

1.24 

2.  Were  the  ratings  easy  or  difficult  to  make? 


(l=very  difficult, 

N 

7=very  easy) 

Mean 

S.D. 

Overall 

123 

3.92 

1.77 

IIB 

54 

3.52 

1.92 

63  B 

33 

4.24 

1.46 

71L 

36 

4.22 

1.73 

NCOS 

72 

3.94 

1.86 

Officers 

41 

4.07 

1.47 

Civilians 

10 

3.10 

2.18 

FORSCOM 

58 

3.83 

1.79 

DOTD 

65 

4.00 

1.77 

3.  How  confident  did  you  feel  about  the  accuracy  of  your  ratings 
(l=not  at  all  confident,  7=very  confident) 


N 

Mean 

S.D. 

Overal 1 

123 

5.74 

1.05 

IIB 

54 

5.72 

1.11 

63B 

33 

5.94 

0.79 

71L 

36 

5.58 

1.16 

NCOS 

72 

5.79 

1.10 

Officers 

41 

5.54 

0.95 

Civilians 

10 

6.20 

0.92 

FORSCOM 

58 

5.79 

1.07 

DOTD 

65 

5.69 

1.0^ 

Table  16  continued 
General  Evaluation 

Which  of  the  following  would  you  change? 

Overall  129  26%  Task  Category  Definitions 

129  15%  Task  Category  Rating  Instructions 

and  Procedures 


Table  17  Job  Activity  Questionnaire  Evaluation 
Job  Activity  Definitions 


1.  How  clear  were  the  job  activity  definitions? 
(l=not  at  all  clear,  7=very  clear) 


N 

Overall  117 


Mean  S.D. 

5.40  1.25 


IIB  43 

63B  35 

71L  39 


5.63 

5.23 

5.31 


1.18 

1.19 

1.38 


NCOS  70 
Officers  42 
Civilians  5 


5.16  1.42 
5.79  0.84 
5.60  0.89 


FORSCOM  65  5.58 

OOTD  52  5.17 


1.21 

1.28 


2. 


How  well  did 
definitions? 


you  understand  the  terms  used  in  the  job  activity 


{l=there  were  many  terms  I  did  not  understand, 
did  not  understand) 


7=there  were  no  terms 


I 


Overal 1 

N 

Mean 

S  D 

117 

6.03 

1.06 

IIB 

43 

5.86 

1.10 

63B 

35 

6.00 

1.21 

71L 

39 

6.23 

0.84 

NCOS 

70 

5.97 

1.19 

Of f i cers 
Civilians 

42 

5 

6.10 

6.20 

0.85 

0.84 

FORSCOM 

65 

6.18 

0  95 

DOTD 

52 

5.83 

1.17 

3.  What  percentage  of  your  job  was  covered  by  these  task  categories? 


Overal 1 

N 

Mean 

s  n 

114 

77.54 

24.87 

IIB 

42 

79.76 

22.68 

636 

35 

77.43 

25.36 

71L 

37 

75.14 

27.14 

NCOS 

68 

71.76 

26.93 

Officers 

41 

85.61 

18.98 

Civilians 

5 

90.00 

17.32 

FORSCOM 

64 

77.97 

24 . 64- 

DOTD 

50 

77.00 

25.41 

Table  17 

continued 

Job  Activity  Ratings 

1.  How 

clear 

were  the 

instructions 

for  maki 

{l=not  at 

all  clear,  7=very  clear) 

N 

Mean 

S.D. 

Overal 1 

114 

6.05 

1.02 

lie 

40 

5.97 

0.95 

63  B 

35 

6.29 

0.79 

71L 

39 

5.92 

1.24 

NCOS 

68 

5.94 

1.12 

Officers 

41 

6.24 

0.86 

Civilians 

5 

6.00 

0.71 

FORSCOM 

62 

6.18 

0.86 

DOTD 

52 

5.90 

1.18 

2 .  Were 

the 

ratings  easy  or  difficult  to  ma 

(l=very  difficult, 

7=very  easy) 

N 

Mean 

S.D. 

Overal 1 

113 

5.12 

1.44 

IIB 

40 

5.17 

1.53 

63B 

34 

4.82 

1.40 

71L 

39 

5.31 

1.36 

NCOS 

67 

5.03 

1.49 

Officers 

41 

5.34 

1.28 

Civilians 

5 

4.40 

1.95 

FORSCOM 

62 

5.42 

1.31 

DOTD 

51 

4.75 

1.51 

3. 


How  confident  did  you  feel  about  the  accuracy  of  your  ratings? 
(l=not  at  all  confident,  7=very  confident) 


N 

Mean 

S.D. 

Overal 1 

114 

5.50 

1.00 

IIB 

40 

5.52 

0.96 

63  B 

35 

5.29 

1.05 

71L 

39 

5.67 

0.98 

NCOS 

68 

5.49 

1.06 

Officers 

41 

5.51 

0.90 

Civilians 

5 

5.60 

1.14 

FORSCOM 

62 

5.61 

1.03 

DOTD 

52 

5.37 

0.95 

Table  17  continued 
General  Evaluation 

Which  of  the  following  would  you  change? 

Overall  117  36%  Job  Activity  Definitions 

117  21%  Job  Activity  Rating  Instructions 

and  Procedures 


Table  18 


Breakdown  of  Subjects  by  Method, 
Rater  Type,  MOS,  and  Command 


Variable 

Cases 

Method 

Importance 

53 

Val idity 

74 

Rater  Type 

Civilian 

11 

NCO 

75 

Officer 

41 

MOS 

IIB 

58 

63B 

32 

71L 

37 

Command 

DOTD 

62 

FORSCOM 

65 

Total 

127 

Table  19 


Rating  of  Performance  Definition  Detail  by 
Method,  Rater  Type,  MOS,  and  Command 

PERFDEFl;  How  would  you  describe  the  amount  of  detail  and  information 


provided  in  the  performance  definition? 

i  W  1  1 

1  2  3 

Far  too  little 

4 

The  right 
amount 

5  6 

Mean 

7 

Far  too  much 

Std  Dev 

Cases 

Summary  for  Total  Sample 

4.55 

.96 

127 

Summary  by  Method 

Importance 

4.58 

.95 

53 

Validity 

4.53 

.98 

74 

Summary  by  Rater  Type 

Civilian 

4.73 

.47 

11 

NCO 

4.55 

1.09 

75 

Officer 

4.51 

.81 

41 

Summary  by  MOS 

IIB 

4.52 

.98 

58 

63B 

4.66 

1.00 

32 

71L 

4.51 

.93 

37 

Summary  by  Command 

DOTD 

4.42 

.97 

62 

FORSCOM 

4.68 

.95 

65 

Table  20 


Rating  of  Ranking  Instruction  Clarity  by 
Method,  Rater  Type,  MOS,  and  Command 


RANKl:  How  clear  were  the  instructions  for  the  ranking? 

1  2  3  4  5  6  7 


Not  at 
all  clear 


Neither  clear 
nor  unclear 


Very 

clear 


Mean  Std  Dev  Cases 

Summary  for  Total  Sample  5.61  1.32  127 


Summary  by  Method 

Importance 

Validity 


Summary  by  Rater  Type 


4.82  1.47  11 
5.53  1.43  75 
5.98  .91  41 


Summary  by  MOS 


5.33  1.50  58 
5.84  1.19  32 
5.86  1.00  37 


Summary  by  Command 


IIB 

63B 

71L 


Civilian 

NCO 

Officer 


6.17  .78  53 

5.22  1.47  74 


DOTD 

FORSCOM 


5.82 

5.42 


1.29 

1.32 


62 

65 


Table  21 


Rating  of  Ranking  Task  Difficulty  by 
Method,  Rater  Type,  MOS,  and  Command 


RANK2:  Was  it  easy  or  difficult  to  rank  order  the  attributes? 


1  2  3 

4  5 

6 

7 

Very 

Neither  easy 

Very 

difficult 

nor  difficult 

easy 

Mean 

Std  Dev 

Cases 

Summary  for  Total  Sample 

4.27 

1.59 

127 

Summary  by  Method 

Importance 

4.51 

1.67 

53 

Val idity 

4.09 

1.52 

74 

Summary  by  Rater  Type 

Civilian 

3.82 

1.33 

11 

NCO 

4.44 

1.69 

75 

Officer 

4.07 

1.46 

41 

Summary  by  MOS 

IIB 

4.31 

1.58 

58 

63B 

3.97 

1.67 

32 

71L 

4.46 

1.54 

37 

Summary  by  Command 

DOTD 

4.31 

1.62 

62 

FORSCOM 

4.23 

1.57 

65 

Table  22 


Rating  of  Confidence  in  Ranking  Accuracy  by 
Method,  Rater  Type,  MOS,  and  Command 


RANKS:  How  confident  do  you  feel  about  the  accuracy  of  your  rankings? 


1  2 

3 

4 

5 

6  7 

Not  at  all 

Somewhat 

Very 

confident 

confident 

confident 

Mean 

Std  Dev 

Cases 

Summary  for  Total  Sample 

4.92 

1.31 

127 

Summary  by  Method 

Importance 

5.24 

1.04 

53 

Validity 

4.69 

1.44 

74 

Summary  by  Rater  Type 

Civilian 

4.91 

.94 

11 

NCO 

4.89 

1.42 

75 

Officer 

4.98 

1.21 

41 

Summary  by  MOS 

IIB 

4.86 

1.28 

58 

63B 

4.69 

1.45 

32 

71L 

5.22 

1.23 

37 

Summary  by  Command 

DOTD 

4.97 

1.33 

62 

FORSCOM 

4.88 

1.30 

65 

Table  23 


Correlations  of  Four  Evaluation  Ratings 
for  Total  Sample  and  Method  Subsamples 

Correlations  for  Total  Sample 


PERFDEFl 

RANKl 

RANK2 

RANK3 

PERFDEFl 

1.0000 

RANKl 

.0500 

1.0000 

RANK2 

.1770 

.3720** 

1.0000 

RANK3 

.0408 

.5152** 

.6523** 

1.0000 

=  127 

p  <  .001 

,  one-tailed 

Correlations 

for  Importance  Subsample 

PERFDEFl 

RANKl 

RANK2  RANKS 

PERFDEFl 

1.0000 

RANKl 

.0712 

1.0000 

RANK2 

.2813 

.2872 

1.0000 

RANK3 

.0664 

.5200** 

.6928**  1.0000 

N  =  53 

*  p  <  .001  ,  one-tailed 


Correlations  for  Validity  Subsample 


PERFDEFl 

RANKl 

RANK2 

RANKS 

PERFDEFl 

RANKl 

RANK2 

RANKS 

1.0000 

.0338 

.0947 

.0205 

1.0000 

.4070** 

.4768** 

1.0000 

.6391** 

1.0000 

N  =  74 

*  p  <  .001 

,  one-tailed 

one-tailed 


Table  24 


ATTDIFF:  Think  about  the  list  of  thirty-one  attributes  (below).  Place 
a  checkmark  in  front  of  any  attributes  that  were  difficult  to  judge  or 
rank.  Please  check  at  least  two  attributes. 


Attribute  Freq 

Verbal  Ability  5 

Reasoning  8 

Number  Ability  10 

Spatial  Ability  35 

Closure  31 

Mental  Information  8 

Processing 

Perceptual  Speed  12 

and  Accuracy 

Memory  3 

Mechanical  Comprehension  8 

Eye-Limb  Coordination  13 

Precision  8 

Movement  Judgment  13 

Hand  and  Finger  Dexterity  10 

Physical  Strength  0 

Physical  Endurance  2 

Balance  and  Flexibility  12 

Involvement  in  Athletics  16 


Attribute  Freq 

Work  Orientation  7 

Sociability  19 

Cooperation/Stability  7 

Energy  3 

Conscientiousness  9 

Dominance/Confidence  17 

Interest  in  Using  8 

Tools  and  Machines 

Interest  in  Rugged  10 

Activities 

Interest  in  Protec-  24 

tive  Services 

Interest  in  Technical  11 

Activities 

Interest  in  Science  23 

Interest  in  Leadership  6 

Interest  in  Artistic  25 

Activities 

Interest  in  Efficiency  13 

and  Organization 


Table  25 


PERFDEF2:  If  you  think  the  performance  definition  has  too  little  detail 
and  information,  what  would  you  add? 


Freq 

% 

No  comment 

116 

91 

Comment 

11 

9 

Total 

127 

100 

Comment 

Freq 

Right  amount  of  detail  2 

Make  examples  specific  (to  MOS)  2 

Best  part  of  today’s  survey  1 

Add  TAMMS  portions  1 

(Add)  Offensive/defensive  details  1 

(Add)  Easy  to  understand  terminology  1 

Make  explanation  clearer  1 

Too  vague--use  scenarios  1 

Closure  is  not  clearly  applicable^  1 

Total  11 


^Comment  refers  to  attribute  definitions,  not  performance  definition. 


Table  26 


PERFDEF3:  If  you  think  the  performance  definition  has  too  much  detail 
and  information,  what  would  you  delete? 


Freq 

% 

No  comment 

112 

88 

Comment 

15 

12 

Total 

127 

100 

Comment 

Freq 

Shorten,  use  fewer  words  6 

All  important  1 

(Delete)  Examples  1 

Rewrite  Core  Technical  Proficiency  (with  1 

regard  to  willingness) 

Write  specific  definitions  and  examples  1 

of  high  and  low  performance 
Too  much  can  be  read  into  this  1 

Merge  some  attributes®  1 

Too  many  attributes®  1 

(Delete)  Spatial  Ability®  1 

Combine  Spatial  Ability  and  Closure®  1 

(Delete)  Interest  in  Technical  Activities®  1 

(Delete)  Interest  in  Artistic  Activities®  1 

(Delete)  Descriptions  of  high,  average,  and  1 

low  levels  on  attributes® 

Movement  Judgment  has  limited  importance®  1 

Total  igb 


®Comment  refers  to  attribute  definitions,  not  performance  definition. 
Some  subjects  provided  more  than  one  comment. 


Table  27 


ATT ADD:  Are  there  any  attributes  you  would  add? 


Freq 

% 

No  comment 

109 

86 

Comment 

18 

14 

Total 

127 

100 

Comment  Freq 

No  additions  necessary  4 

Temperament/Attitude 

Integrity/Judgment/Moral  Discretion  2 

Coping  with  Stress  1 

Replace  Work  Orientation  with  1 

Dedication  to  Duty 

Military  Bearing  1 

More  temperament  attributes  1 

Loyalty  to  Country  1 

Optimism  1 

Value  of  Human  Life  1 

Sense  of  Humor  1 

Interests 

Interest  in  Values  1 

Interest  in  Family  1 

Interest  in  Writing  1 

Interest  in  Business  1 

Interest  in  History  1 

Cognitive  Abilities 

Attention  to  Detail  1 

Ability  to  Communicate  (e.g.,  bilingual)  1 

Reading  Comprehension  1 

Research  Ability  1 

Coordination  &  Physical  Abilities 

Team  Sports  Player  1 

Team  Sports  Captain  1 

Miscellaneous 

Religion  1 

Ability  to  Kill  (mentally)  1 

Total  27® 


®Some  subjects  provided  more  than  one  comment. 


Table  28 


ATTDEL:  Are  there  any  attributes  you  would  delete? 

Freq  % 

No  comment  99  78 

Comment  28  22 

Total  127  100 

Comment 

Yes,  some  attributes  should  be  deleted  4 

Interests 

Interest  in  Artistic  Activities  16 

Interest  in  Science  6 

Interest  in  Protective  Services  2 

Interest  in  Rugged  Activities  2 

Interest  in  Technical  Activities  1 

Coitive  Abilities 

Closure  3 

Spatial  Ability  1 

Combine  Reasoning,  Spatial  Ability,  1 

Closure,  and  Mental  Information 
Processing  into  one  attribute 

Temperament/Attitude 

Cooperation  (need  only  Stability)  1 

Dominance  (need  only  Confidence)  1 

Coordination  &  Physical  Abilities 

Involvement  in  Athletics  2 

Total  40a 

®Some  subjects  provided  more  than  one  comment. 


Table  29 


General  Evaluation  Summary 

GENERAL  EVALUATION:  Think  about  the  two  tasks  you  completed,  attribute 
judgments  and  ranking. 


GENEVALl:  Which  was  easier? 


Freq 

% 

Judgment 

63 

50 

Ranking 

51 

40 

Both® 

2 

2 

Neither® 

11 

9 

Total 

127 

lOl*^ 

GENEVAL2:  Which  provides  more  information  about  the 

MOS? 

Freq 

% 

Judgment 

80 

63 

Ranking 

35 

28 

Both® 

2 

2 

Neither® 

10 

8 

Total 

127 

101*^ 

GENEVAL3:  If  the  Army  was 

going  to 

use  one  of  these  methods  to  guide 

selection  and  placement  of 

soldiers 

in  MOS,  which  would  you  prefer? 

Freq 

% 

Judgment 

72 

57 

Ranking 

42 

33 

Both® 

3 

2 

Neither® 

10 

8 

Total 

127 

100 

®This  response  option  was  not  provided  on  the  form;  some 
subjects  added  it. 

^Percentages  do  not  sum  to  100  due  to  rounding  error. 


Table  30.  Cross-Tabulation  of  MOS  and  Rater  Type  for  Importance  Rating  Method 


Rater  Type 


NCQ 

Officer 

Civilian 

Total 

MOS 

IIB 

21 

2 

1 

24 

63B 

4 

7 

3 

14 

71L 

17 

16 

1 

34 

Total 

42 

25 

5 

72 

Table  31.  Cross-Tablulation  of  MOS  and  Rater  Type  for  Validity  Rating  Method 


Rater  Type 


NCQ 

Officer 

Civilian 

Total 

MOS 

IIB 

16 

16 

6 

38 

63B 

18 

6 

0 

24 

71L 

16 

12 

1 

29 

Total 

50 

34 

7 

91 

Table  32.  ANOVA  of  Validity  Ratings  for  Core  Technical  Performance  Area 


Source 

MS 

F 

a 

Between  Subjects: 

MOS 

2 

148.58 

5.34 

<.01 

Type 

1 

13.74 

0.49 

0.49 

MOS  X  Type 

2 

14.47 

0.52 

0.60 

Error 

65 

27.84 

Within  Subjects: 

Attributes 

30 

54.51 

23.31 

<.001 

Attributes  x  MOS 

60 

19.85 

8.49 

<.001 

Attributes  x  Type 

30 

3.50 

1.50 

0.04 

Attributes  x  MOS  x  Type 

60 

5.78 

■  2.47 

<.001 

Error 

1950 

2.34 

Note:  Type  includes  NCOs  and  Officers  only. 


Table  33.  ANOVA  of  Validity  Ratings  for  General  Soldiering  Performance  Area 


Source 

df 

MS 

F 

a 

Between  Subjects: 

MOS 

2 

65.29 

2.04 

0.14 

Type 

1 

34.67 

1.08 

0.30 

MOS  X  Type 

2 

25.56 

.80 

0.46 

Error 

65 

32.07 

Within  Subjects: 

Attributes 

30 

42.51 

20.97 

<.001 

Attributes  x  MOS 

60 

2.86 

1.41 

.02 

Attributes  x  Type 

30 

4.04 

1.99 

<.001 

Attributes  x  MOS  x  Type 

60 

3.32 

1.64 

<.01 

Error 

1950 

2.03 

Note;  Type  includes  NCOs  and  Officers  only. 


Table  34.  ANOVA  of  Validity  Ratings  for  Effort/Leadership  Performance  Area 


Source 

MS 

F 

B. 

Between  Subjects: 

MOS 

2 

262.75 

9.72 

<.001 

Type 

1 

1.70 

.06 

0.80 

MOS  X  Type 

2 

38.88 

1.44 

0.25 

Error 

65 

27.04 

Within  Subjects: 

Attributes 

30 

79.88 

34.01 

<.001 

Attributes  x  MOS 

60 

2.45 

1.04 

0.39 

Attributes  x  Type 

30 

8.26 

3.52 

<.001 

Attributes  x  MOS  x  Type 

60 

4.82 

2.05 

<.001 

Error 

1950 

2.35 

Note:  Type  includes  NCOs  and  Officers  only. 


Table  35.  ANOVA  of  Validity  Ratings  for  Personal  Discipline  Performance  Area 


Source 

df 

MS 

F 

B. 

Between  Subjects: 

MOS 

2 

286.18 

6.82 

<.01 

Type 

1 

177.75 

4.24 

0.04 

MOS  X  Type 

2 

297.76 

7.10 

<.01 

Error 

64 

41.96 

Within  Subjects: 

Attributes 

30 

70.13 

27.03 

<.001 

Attributes  x  MOS 

60 

3.25 

1.25 

0.09 

Attributes  x  Type 

30 

8.43 

3.25 

<.001 

Attributes  x  MOS  x  Type 

60 

3.13 

1.21 

0.14 

Error 

1920 

2.59 

^te:  Type  includes  NCOs  and  Officers  only. 


Table  36.  ANOVA  of  Validity  Ratings  for  Physical  Fitness/Military  Bearing 
Performance  Area 


Source 

df 

MS 

F 

B. 

Between  Subjects: 

MOS 

2 

188.67 

4.70 

<.01 

Type 

1 

230.43 

5.74 

0.02 

MOS  X  Type 

2 

251.47 

6.27 

<.01 

Error 

65 

40.13 

Within  Subjects: 

Attributes 

30 

136.99 

46.24 

<.001 

Attributes  x  MOS 

60 

4.38 

1.48 

<.01 

Attributes  x  Type 

30 

8.65 

2.92 

<.001 

Attributes  x  MOS  x  Type 

60 

4.87 

1.64 

<.01 

Error 

1950 

2.96 

Note:  Type  includes  NCOs  and  Officers  only. 


Table  37.  ANOVA  of  Validity  Rankings 


Source  ^ 

Between  Subjects: 

MOS  2 

Type  1 

MOS  X  Type  2 

Error  61 

Within  Subjects: 

Attributes  30 

Attributes  x  MOS  60 

Attributes  x  Type  30 

Attributes  x  MOS  x  Type  60 

Error  1830 


MS 

F 

B. 

1.18 

1.12 

0.33 

0.86 

0.82 

0.37 

1.16 

1.10 

0.34 

1.05 

1331.13 

29.41 

<.001 

382.76 

8.46 

<.001 

70.76 

1.56 

0.03 

57.33 

1.27 

0.08 

45.26 

Note:  Type  includes  NCOs  and  Officers  only. 


Table  38.  Reliabilities  of  Validity  Ratings  by  MOS,  Rater  Type,  and  Criterion 


Performance  Area 


/yl 

IIB 

NCOS 

15 

Officers 

13 

NCOS  &  Officers 

28 

63B 

NCOS 

17 

Officers 

6 

NCOS  &  Officers 

21 

71L 

NCOS 

12 

Officers 

10 

NCOS  &  Officers 

22 

Core 

General 

Eff 

.76 

.75 

.79 

.78 

.83 

.94 

.76 

00 

.88 

.79 

.73 

.81 

N/A 

N/A 

N/A 

in 

00 

• 

.79 

.84 

.93 

.86 

.84 

.94 

.86 

.91 

.93 

.84 

.87 

Disc 

Fit 

RNK- 

5 

.73 

.89 

.92 

(14) 

.93 

.95 

.93 

(15) 

• 

00 

cn 

.92 

.93 

(29) 

.77 

.84 

.77 

(13) 

N/A 

N/A 

N/A 

(3) 

.81 

.88 

.83 

(16) 

.81 

.89 

.96 

(13) 

.91 

.94 

.95 

(10) 

.87 

.91 

.95 

(23) 

Note;  Reliabilities  adjusted  to  X  -  15. 

®  These  sample  sizes  refer  to  the  performance  area  analyses. 
^  Sample  sizes  for  the  ranking  analyses  are  in  parentheses. 


Table  39.  Reliabilities  of  Importance  Ratings  by  MOS,  Rater  Tyoe. 
and  Criterion 


fi¬ 

ne  NCOS  17 

Officers  2 

NCOS  &  Officers  18 

63B  NCOS  4 

Officers  7 

NCOS  &  Officers  8 

71L  NCOS  16 

Officers  14 

NCOS  &  Officers  30 


Performance  Area 


Core 

General 

Eff 

.89 

.89 

.86 

N/A 

N/A 

N/A 

.90 

.90 

.87 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

.91 

.88 

.91 

.95 

.87 

.91 

.95 

.89 

.92 

.95 

.87 

.91 

Disc 

Fit 

RNK^ 

.83 

.92 

.94 

(20) 

N/A 

N/A 

N/A 

(1) 

.84 

.93 

.94 

(21) 

N/A 

N/A 

N/A 

(3) 

N/A 

N/A 

N/A 

(5) 

.86 

.91 

.94 

(8) 

.85 

.91 

.96 

(13) 

.88 

.95 

.96 

(13) 

.87 

.93 

.96 

(26) 

Note:  Reliabilities  adjusted  to  K  =  15. 

These  sample  sizes  refer  to  the  performance  area  analyses. 
^  Sample  sizes  for  the  ranking  analyses  are  in  parentheses. 


Table  40.  Correlations  of  Mean  Importance  and  Mean  Validity  Ratings  by  MOS 
for  Core  Technical  Proficiency,  Computed  Across  the  31  Attributes 


IIB  Imo 

63B  Imo 

71 L  Imo  IIB  Val 

63B  Val  71L  Val 

IIB  Imp 

1.00 

63B  Imp 

.41 

1.00 

71L  Imp 

.38 

.34 

1.00 

IIB  Val 

.81 

.51 

.45  1.00 

63B  Val 

.45 

.88 

.35  .52 

1.00 

71L  Val 

.39 

.37 

.98  .49 

.35  1.00 

Note:  Mean  ratings  are 

based  on 

the  combined  NCO  and  Officer  groups. 

Table  41. 

Correlations 
for  General 

of  Mean  Importance  and  Mean  Validity  Ratings  by  MOS 
Soldier  Proficiency,  Computed  Across  the  31  Attributes 

llB  Imp 

63B  Imo 

71 L  Imo 

IIB  Val 

63_B  Val  71 L  Val 

IIB  Imp 

1.00 

63B  Imp 

.71 

1.00 

71L  Imp 

.81 

.87 

1.00 

IIB  Val 

.80 

.84 

.90 

1.00 

63B  Val 

.75 

.83 

.85 

.87 

1.00 

71L  Val 

.82 

.82 

.89 

00 

.78  1.00 

Note:  Mean  ratings  are  based  on  the  combined  NCO  and  Officer  groups. 


Table  42.  Correlations  of  Mean  Importance  and  Mean  Validity  Ratings  by  MOS 
for  Effort  and  Leadership,  Computed  Across  the  31  Attributes 


IIB  Imo 

63B  Imo 

71L  Imo 

IIB  Val 

63B  Val 

IIB  Imp 

1.00 

63B  Imp 

.71 

1.00 

71L  Imp 

.72 

.92 

1.00 

IIB  Val 

.80 

.87 

.93 

1.00 

63B  Val 

.77 

.90 

.89 

.93 

1.00 

71L  Val 

.81 

.86 

.92 

.92 

.86 

Note:  Mean  ratings  are  based  on  the  combined  NCO  and  Officer  groups. 


Table  43,  Correlations  of  Mean  Importance  and  Mean  Validity  Ratings  by 
MOS  for  Personal  Discipline,  Computed  Across  the  31  Attributes 


IIB  Imo 

63B  Imo- 

71L  Imo 

IIB  Val 

IIB  Imp 

1.00 

63B  Imp 

.81 

1.00 

71L  Imp 

.77 

.85 

1.00 

IIB  Val 

.85 

.86 

.87 

1.00 

63B  Val 

.81 

.89 

.89 

.92 

71L  Val 

.83 

.86 

.93 

.89 

63B  Val  71L  Val 


1.00 

.90  1.00 


cer  groups. 


Note:  Mean  ratings  are  based  on  the  combined  NCO  and  Offi 


Table  44.  Correlations  of  Mean  Importance  and  Mean  Validity  Ratings  by 
MOS  for  Physical  Fitness/Military  Bearing,  Computed  Across 
the  31  Attributes 


IIB  Imo 

63B  Imp 

71L  Imo  IIB  Val 

63B  Val 

71L  Val 

IIB  Imp 

1.00 

63B  Imp 

.89 

1.00 

71L  Imp 

.95 

.94 

1.00 

IIB  Val 

.94 

.89 

.96  1.00 

63B  Val 

.91 

.92 

.94  .94 

1.00 

71L  Val 

.93 

p 

.91 

.95  .91 

.91 

1.00 

Note:  Mean  ratings  are 

based  on 

the  combined  NCO  and  Officer 

groups. 

Table  45. 

Correlations  of  Mean  Importance  and  Mean  Validity  Ranks 

Computed  Across  the  31  Attributes 

IIB  Imo 

63B  Imo 

71L  Imp 

IIB  Val 

63B  Val  71 L  Val 

IIB  Imp 

1.00 

63B  Imp 

.20 

1.00 

71L  Imp 

.60 

.27 

1.00 

IIB  Val 

.95 

.31 

.56 

1.00 

63B  Val 

.48 

.84 

.53 

.54 

1.00 

71 L  Val 

.60 

.29 

.98 

.55 

.54  1.00 

Note:  Mean  ranks  are  based  on  the  combined  NCO  and  Officer  groups. 


Table  46.  Correlations  of  Mean  Importance  and  Mean  Validity  Ratings, 

Computed  Across  the  155  Attribute  x  Performance  Area  Combinations 


IIB  Imp 

IIB  Imo 

1.00 

63B  Imo 

63B  Imp 

.75 

1.00 

71L  Imp 

.75 

.77 

IIB  Val 

.88 

.81 

63B  Val 

.78 

.89 

71L  Val 

.77 

.76 

71L  Imo 

IIB  Val 

63B  Val 

71L  Val 

1.00 

.82 

1.00 

.79 

.87 

1.00 

.95 

.81 

.76 

1.00 

Note;  Mean  ratings  are  based  on  the  combined  NCO  and  Officer 


groups. 


Table  47.  Correlations  of  Mean  Importance  and  Mean  Validity  Rankings 

with  Mean  Importance  and  Mean  Validity  Ratings  in  Each  of  the 
Five  Performance  Areas  for  IIB;  Means  are  based  on  NCO  and 
Officer  groups. 


Mean  Importance: 


Core  Technical 
General  Soldier 
Effort/Leadership 
Personal  Discipline 
Fitness/Bearing 

Mean  Validity: 


Core  Technical 

.755 

.789 

General  Soldier 

.754 

.815 

Effort/Leadership 

.627 

.685 

Personal  Discipline 

.473 

.564 

Fitness/Bearing 

.568 

.630 

Ng^:  N  =  31  for  each  of  the  correlations  of  means  (number  of  attributes). 
Correlations  have  been  appropriately  reflected. 


.864 

.827 

.868 

.843 

.823 

.787 

.790 

.808 

.574 

.613 

Table  48.  Correlations  of  Mean  Importance  and  Mean  Validity  Rankings 

with  Mean  Importance  and  Mean  Validity  Ratings  in  Each  of  the 
Five  Performance  Areas  for  63B:  Means  are  based  on  NCO  and 
Officer  groups. 


Mean  Importance  Ranks  Mean  Validity  Ranks 


Mean  Importance: 

Core  Technical 

.919 

.835 

General  Soldier 

.344 

.473 

Effort/Leadership 

.086 

.201 

Personal  Discipline 

.166 

.294 

Fitness/Bearing 

.178 

.225 

Mean  Validity: 

Core  Technical 

.920 

.813 

General  Soldier 

.310 

.329 

Effort/Leadership 

.070 

.152 

Personal  Discipline 

.013 

.056 

Fitness/Bearing 

-.004 

.006 

Mots:  N  =  31  for  each  of  the  correlations  of  means  (number  of  attributes). 
Correlations  have  been  appropriately  reflected. 


Table  49.  Correlations  of  Mean  Importance  and  Mean  Validity  Rankings 

with  Mean  Importance  and  Mean  Validity  Ratings  in  Each  of  the 
Five  Performance  Areas  for  71L:  Means  are  based  on  NCO  and 
Officer  groups. 


Mean  Importance: 


Core  Technical 
General  Soldier 
Effort/Leadership 
Personal  Discipline 
Fitness/Bearing 

Mean  Validity: 


Core  Technical 

.944 

.952 

General  Soldier 

.572 

.569 

Effort/Leadership 

.659 

.591 

Personal  Discipline 

.528 

.461 

Fitness/Bearing 

-.097 

-.083 

.952 

.955 

.657 

.633 

.670 

.606 

.604 

.534 

-.027 

-.028 

Nflte:  N  -  31  for  each  of  the  correlations  of  means  (number  of  attributes). 
Correlations  have  been  appropriately  reflected. 


Table  50.  Mean  Importance  Ratings  of  Attributes  for  Core  Technical  Proficiency: 
IIB,  63B,  and  71L  NCOS  and  Officers  Combined 
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Table  51.  Mean  Importance  Ratings  of  Attributes  for  General  Soldiering  Proficiency 
IIB,  63B,  and  71L  NCOS  and  Officers  Combined 
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Table  52,  Mean  Importance  Ratings  of  Attributes  for  Effort  and  Leadership: 
IIB,  63B,  and  71L  NCOS  and  Officers  Combined 
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Table  53.  Mean  Importance  Ratings  of  Attributes  for  Personal  Discipline: 
IIB,  63B,  and  71L  NCOS  and  Officers  Combined 
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Table  54.  Mean  Importance  Ratings  of  Attributes  for  Physical  Fitness/Military  Bearing: 
IIB,  63B,  and  71L  NCOS  and  Officers  Combined 
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Table  55.  Mean  Importance  Rankings  of  Attributes: 

IIB,  63B,  and  71L  NCOS  and  Officers  Combined 


VO  VO  VO  VO  VO  VO  VO  VO  VO  VO  VO  VO  VO  VO  VO  VO  VO  VOVOVOVOVOVO  VO  VO  VO  VO  VO  VO  VO  VO 
CN  <N  <N  CN  03  OJ  OJ  ca  CM  CM  CM  CM  CM  CM  CM  CM  OJ  CM03CMCMCMCM  OJ  03  CM  CM  CM  CM  CM  CM 


H  TJ 

w 


VO  cn CO  cn  00  If) H  CM  ^  <T»  VO  CJ)  cn  O  m  O^O^O^^CO^^  ^CTiCnVOlOVOrHOO 
VO  o*  VO  CO  o  CO  CO  inHcococMvovoin  co^^r^in^  vot^'^’io^r^c^m 


CQ 

CO  03 
VO  ^ 
CO 


H  03 
H 


cMnvovot^cnvo-«!ftr) 

CO  in  CO  VO  m  in  ^ 

^voinM*M*in 

^  CO  M*  00 in 

o  cn  CO  o>  CO  03  0- o 
in  c^  CO  H  CO  o>  c^  CM  in 

H  CM 'ct  H  VO  in  in  o 
00  (D  O  00  ^  VO  CO  in 

(7)  in -M* 

r^Ht^c^coin 

H  CO  M*  00 00  O  CO 
CO  CM  in  O  O  CO  O  CM 

coincovoHino^r^cM 

Hf-I  H  CM 

^  VO  CM  0)  C^  CO  CO  ^ 
Hi— ICM  1— IHCMCM 

voM*o)cor^in 

HHH 

invoinHc^t^oocM 

CM  CM  CM  CM  CM  H  CM  H 

00000)00(7100(7)0)00 

(3)  00  C7)  (7)  00  00  0)  00 

00(7)00(7)00(7) 

0)00(7)00(7)00  00(7) 

COHr^O*^COCOH(J) 

^CMVOCOVOCMCMr^CO 

O  CM  CO  (7)  CM  O 
O  O  CO  00  (7)  CM  CO  in 

r^CMHr^Ht^ 
r^Hvoin  voco 

CM  in  CM  H  (7)  CO  ^  VO 

ovocor^voo)f^0) 

in  VO  VO  VO  VO  00  CO  H 

00  VO  CO  VO  ^  in 

r^r^voint^r* 

CO  00  h- VO  CO  o  00 

ococoin^in’^oco 

OCOr^CM^t^’^tOH 

•  •••••••• 

o  VO  o  o  VO  in 
VO  in  in  VO  o  in  in 

oot^ot^covo 

oovoinvoHin 

HOO)OOM*OOOH 

HOOOCOM*OCOH 

0‘CMO)(ncMinvoo<M 
H  H  H  CMH  HH 

o  CM  VO  00  CO in  in 

HHH  HHHCM 

OOCOOOCMMT^ 
CM  H  H  H  H 

■M*  C>«  r*- CO  Tf  o  M* 

HCM  CMCMCOH 

HrHiHr-tc-HHHHH 
CM  CM  CM  CM  CM  CM  CM  CM  CM 

HHHHHHHH 
(M  CM  CM  CM  CM  CM  CM  (M 

H  H  H  H  H  H 
CM  CM  CM  CM  (M  CM 

HHHHHHHH 
CM  CM  CM  CM  CM  CM  CM  CM 

HHinino^co^cj) 
CO  H  0)  CO  O' 00  00  CM  00 

00  00  H  (7)  VO  CO 

00  CO  CO  0)  O  H  in  H 

00CMM*r^OH 

inin^incM^ 

(7)  00  O  ^  O  CM 

OHcoinco^HCM 

vooovoc^’^ininvo^ 

in  in  CM  VO  0) 

VO  00  00 00  VO 

in  (7)  VO  CO  ^  CM 

o  CO  o  VO  CM  in  VO  in  in 
o  CO  H  r- in  (7i  r- 0)  (3) 

in  VO  CO  VO  ^  H  (7)  in 
OCOCOr-Ht^HO 

Or^CMVOOCT) 

Hvoinr^HCM 

oinocooo^M* 

O0)HC0H0)HH 

C^  CO  H  CO  CM  00  CO  0)  H 
CMHCM  H  CM 

CM  CM  in  in  VO  CO  CO  cTi 

H  H  H  H  H  H 

CO  Tt  CO  CO  CO 
H  H  H  H  H  H 

in  VO  CO  VO  00  CM  O  O 
CM  H  CM  CM  CM  H  CO  CM 

m 

CO 

0) 

0  a 

P  s 

H  0)  ^  O 
HCU((iUCU  >iO 
(3  O  (U-H  M 

^COfl4JCOO  ox: 
U  (0  g  (3  0<H|<  s  o 
(1)  q  d  0^  cto  0)  0) 
>P!jawuHa.gg 


TJ  « 

M  O' 

O  *0 
O  3 
U  ITJ 

C  >1  Q)  (0 
AO-P-PAU  u 
S*H  C'HJJ  C<  ffl'H 
•H  CO  «  M  O'nJ  O-P 

g  e  c  p  c  a 

I  Oe4J«3flJH 
e  e  >  X  P-Ornr! 
>iM  O  (U-4J  C  «J4J 
H04SQC0H(n<< 


-P 

0 

•P  >iC 

0 

<0  0 

0  u 

C-P  o 

•H 

CQ 

0)  (U 

c  ® 

0) 

iH 

tTLpx:  eTj 

•HiH-P 

c 

0 

0 

b'O  o*H  flj 

P*H  n) 

0) 

c 

0 

3  P  ®  u  ® 

0X5  U  >fH 

(3 

HtOhJ 

<3  0)  CPO 

c 

.«-H  ap  0). 

’H 

• 

•  • 

•  •  • 

p  u  o  e  c  g 
o  o  o  c  o  o 

ISCQUHOQ 


4J4J4J-P+J4J-P4J 

cccccccc 

HHHHHHHH 


Table  56,  Mean  Validity  Ratings  of  Attributes  for  Core  Technical  Proficiency; 
IIB,  63B,  and  71L  NCOS  and  Officers  Combined 
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Table  57.  Mean  Validity  Ratings  of  Attributes  for  General  Soldiering  Proficiency: 
IIB,  63B,  and  71L  NCOS  and  Officers  Combined 
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Table  58.  Mean  Validity  Ratings  of  Attributes  for  Effort  and  Leadershio 
IIB,  63B,  and  71L  NCOS  and  Officers  Combined 
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Table  59.  Mean  Validity  Ratings  of  Attributes  for  Personal  Discipline 
IIB,  63B,  and  71L  NCOS  and  Officers  Combined 
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Table  60.  Mean  Validity  Ratings  of  Attributes  for  Physical  Fitness/Military  Bearing 
IIB,  63B,  ana  71L  NCOS  and  Officers  Combined 
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Table  61.  Mean  Validity  Rankings  of  Attributes: 

IIB,  63B,  ana  71L  NCOs  and  Officers  Combined 
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APPENDIX  A 


DEMOGRAPHIC  INFORMATION  OF  SMEs 


Appendix  A 

Sample  Demographic  Information 


MOS 

FREQUENCY 

PERCENT 

CUMULATIVE 

FREQUENCY 

CUMULATIVE 

PERCENT 

IIB 

60 

44.8 

60 

44.8 

63B 

35 

26.1 

95 

70.9 

71L 

39 

29.1 

134 

100.0 

RANK 

FREQUENCY 

PERCENT 

CUMULATIVE 

FREQUENCY 

CUMULATIVE 

PERCENT 

NCO 

77 

57.5 

77 

57.5 

OFFICER 

46 

34.3 

123 

91.8 

CIVILIAN 

11 

8.2 

134 

100.0 

CUMULATIVE 

CUMULATIVE 

COMMAND 

FREQUENCY 

PERCENT 

FREQUENCY 

PERCENT 

FORSCOM 

66 

49.3 

66 

49.3 

DOTD 

68 

50.7 

134 

100.0 

. 

CUMULATIVE 

CUMULATIVE 

HGRADE 

FREQUENCY 

PERCENT 

FREQUENCY 

PERCENT 

3 


E4 

1 

0.8 

1 

0.8 

E5 

11 

8.4 

12 

9.2 

E6 

20 

15.3 

32 

24.4 

E7 

38 

29.0 

70 

53.4 

E8 

12 

9.2 

82 

62.6 

E9 

1 

0.8 

83 

63.4 

1ST  LT 

4 

3.1 

87 

66.4 

2ND  LT 

20 

15.3 

107 

81.7 

CAPTAIN 

20 

15.3 

127 

96.9 

MAJOR 

3 

2.3 

130 

99.2 

LT  COL 

1 

0.8 

131 

100.0 

SITE 

CUMULATIVE 

CUMULATIVE 

FREQUENCY  PERCENT  FREQUENCY 

PERCENT 

ORD 

28  20.9 

28 

20.9 

BENJAMIN 

HARRISON 

21  15.7 

49 

36.6 

BENNING 

26  19.4 

75 

56.0 

ABERDEEN 

21  15.7 

96 

71.6 

STEWART 

38  28.4 

134 

100.0 

A1 


Sample  Demographic  Information 
TABLE  OF  HGRADE  BY  MOS 


HGRADE  MOS 


FREQUENCY 

PERCENT 

ROW  PCT 

COL  PCT 

IIB 

63B  1  71L  1 

L  _1_  J 

E4 

1 

0.76 

100.00 

1.69 

r  n 

0 

0.00 

0.00 

0.00 

L  J 

r  - 

0 

0.00 

0.00 

0.00 

L-  -  J 

E5 

9 

6.87 

81.82 

15.25 

0 

0.00 

0.00 

0.00 

L  J 

E6 

9 

6.87 

45.00 

15.25 

r  - 

9 

6.87 

45.00 

27.27 

L«  -  _ _ 

”  "  "  '“'1 

E7 

r1 

17 

12.98 

44.74 

28.81 

L-  _J 

P  "  "  "  “1 

10 

7.63 

26.32 

30.30 

L..  -  .  -  >  -  .  -  J 

r  -  - 

11 

8.40 

28.95 

28.21 

L  _1 

E8 

4 

3.05 

33.33 

6.78 

1 

0.76 

8.33 

3.03 

L  _  _  J 

7 

5.34 

58.33 

17.95 

L  J 

E9 

0 

0.00 

0.00 

0.00 

L  J 

0 

0.00 

0.00 

0.00 

L  J 

1 

0.76 

100.00 

2.56 

TOTAL  59  33  39 

45.04  25.19  29.77 

(CONTINUED)  . 


A2 


TOTAL 

1 

0.76 


11 

8.40 


20 

15.27 


38 

29.01 


12 

9.16 


1 

0.76 


131 

100.00 


Sample  Demographic  Information 
TABLE  OF  HGRADE  BY  MOS 
HGRADE  MOS 


FREQUENCY 

PERCENT 

ROW  PCT 

COL  PCT 

IIB  I 

1  63B  1  71L 

L 

1ST  LT 

3 

2.29 

75.00 

5.08 

L«  «J 

r  ^  ”  - : 

0 

0.00 

0.00 

0.00 

L  _  _  _ 1 

r - - 

1 

0.76 

25.00 

2.56 

L 

2ND  LT 

11 

8.40 

55.00 

18.64 

5 

3.82 

25.00 

15.15 

L- _ 1 

4 

3.05 

20.00 

10.26 

L 

CAPTAIN 

4 

3.05 
20.00 
_  6.78 

5 

3.82 

25.00 

15.15 

L _  J 

11 

8.40 

55.00 

28.21 

L 

MAJOR 

0 

0.00 

0.00 

0.00 

1 

0.76 

33.33 

3.03 

1 _  J 

2 

1.53 

66.67 

5.13 

L 

LT  COL 

r“  "  ~  "1 

1 

0.76 

100.00 

1.69 

L-  J 

0 

0.00 

0.00 

0.00 

L  ..  1 

0 

0.00 

0.00 

0.00 

TOTAL 

59  '  33  '  39 

45.04  25.19  29.77 

FREQUENCY  MISSING  =  3 


A3 


TOTAL 

4 

3.05 


20 

15.27 


20 

15.27 


3 

2.29 


1 

0.76 


131 

100.00 


Sample  Demographic  Information 


TABLE  OF  RANK  BY  COMMAND 
RANK  COMMAND 


FREQUENCY 

PERCENT 

ROW  PCT 

COL  PCT 

FORSCOM 

L  J 

DOTD 

1 _ 

NCO 

35 

26.12 

45.45 

53.03 

L—  _  _  _ J 

42 

31.34 

54.55 

61.76 

OFFICER 

31 

23.13 

67.39 

46.97 

15 

11.19 

32.61 

22.06 

1 _ 

CIVILIAN 

P  “  “  ~  “  1 

0 

0.00 

0.00 

0.00 

L  J 

11 

8.21 

100.00 

16.18 

L 

TOTAL  66  68 

49.25  50.75 

TOTAL 

77 

57.46 


46 

34.33 


11 

8.21 


134 

100.00 
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Sample  Demographic  Information 
TABLE  OF  HGRADE  BY  COMMAND 


HGRADE  COMMAND 


FREQUENCY 
PERCENT 
ROW  PCT 
COL  PCT 


FORSCOM 


DOTD 


- - --H 

E4 

h - H 

1 

0.76 

100.00 

1.52 

_  _j 

h-- . + 

0 

0.00 

0.00 

0.00 

L  J_ 

E5 

10 

7.63 

90.91 

15.15 

1 

0.76 

9.09 

1.54 

L  _  J_ 

E6 

5 

3.82 

25.00 

7.58 

15 

11.45 

75.00 

23.08 

L  —  _l_ 

E7 

r“  “I 

18 

13.74 

47.37 

27.27 

-  J 

20 

15.27 

52.63 

30.77 

L-.  _ 

E8 

2 

1.53 

16.67 

3.03 

10 

7.63 

83.33 

15.38 

L  _l_ 

m 

lO 

L  J 

0 

0.00 

0.00 

0.00 

L  J 

1 

0.76 

100.00 

1.54 

L  _L 

TOTAL  '  66 

50.38 

65 

49.62 

(CONTINUED) 


TOTAL 

1 

0.76 


11 

8.40 


20 

15.27 


38 

29.01 


12 

9.16 


1 

0.76 


131 

100.00 
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Sample  Demographic  Information 
TABLE  OF  HGRADE  BY  COMMAND 
HGRADE  COMMAND 


FREQUENCY 

PERCENT 

ROW  PCT 

COL  PCT 

FORSCOM 

L _ _ 

DOTD 

L 

1ST  LT 

4 

3.05 

100.00 

6.06 
......  .j 

r-  —  -  --  - 

0 

0.00 

0.00 

0.00 

L  - 

2ND  LT 

14 

10.69 

70.00 

21.21 

U-  ......  .J 

6 

4.58 

30.00 

9.23 

L 

CAPTAIN 

12 

9.16 

60.00 

18.18 

U.  .  -  .  .  J 

8 

6.11 

40.00 

12.31 

L 

MAJOR 

0 

0.00 

0.00 

0.00 

L _  J 

3 

2.29 

100.00 

4.62 

L 

LT  COL 

0 

0.00 

0.00 

0.00 

_1 

1 

0.76 

100.00 

1.54 

TOTAL 

66  65 

50.38  49.62 

FREQUENCY  MISSING  =  3 


TOTAL 

4 

3.05 


20 

15.27 


20 

15.27 


3 

2.29 


1 

0.76 


131 

100.00 
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Sample  Demographic  Information 
TABLE  OF  HGRADE  BY  RANK 
HGRADE  RANK 


FREQUENCY 

PERCENT 

ROW  PCT 

COL  PCT 

NCO 

L _ 

OFFICER 

1 _  _ _ 1 

CIVILIAN 

L 

E4 

r  i 

1 

0.76 

100.00 

1.30 

L _  -  J 

0 

0.00 

0.00 

0.00 

L  J 

0 

0.00 

0.00 

0.00 

L_  -  _ 

E5 

■ni»m  M 

■mg  a 

0 

0.00 

0.00 

0.00 

L 

E6 

r  n 

20 

15.27 

100.00 

25.97 

L.... _ 1 

r  -  1 

0 

0.00 

0.00 

0.00 

L _  J 

0 

0.00 

0.00 

0.00 

L 

E7 

37 

28.24 

97.37 

48.05 

L«  «J 

1 

0.76 

2.63 

2.17 

L—  J 

0 

0.00 

0.00 

0.00 

L 

E8 

7 

5.34 

58.33 

9.09 

0 

0.00 

0.00 

0.00 

L«-. _ 1 

5 

3.82 

41.67 

62.50 

L 

E9 

1 

0.76 

100.00 

1.30 

L  J 

r  1 

0 

0.00 

0.00 

0.00 

L  ...  1 

0 

0.00 

0.00 

0.00 

L 

TOTAL 

77  46 

58.78  35.11 

8 

6.11 

(CONTINUED) 
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TOTAL 

1 

0.76 


11 

8.40 


20 

15.27 


38 

29.01 


12 

9.16 


1 

0.76 


131 

100.00 


Sample  Demographic  Information 
TABLE  OF  HGRADE  BY  RANK 


HGRADE  RANK 


FREQUENCY 

PERCENT 

ROW  PCT 

COL  PCT 

NCO 

L_  ^  ^  .  J 

OFFICER 

CIVILIAN 

L  _L 

1ST  LT 

P  I 

0 

0.00 

0.00 

0.00 

_ 

4 

3.05 

100.00 

8.70 

L_ 

0 

0.00 

0.00 

0.00 

L  _l_ 

2ND  LT 

0 

0.00 

0.00 

0.00 

18 

13.74 

90.00 

39.13 

_  _  j 

2 

1.53 

10.00 

25.00 

L  1 

CAPTAIN 

0 

0.00 

0.00 

0.00 

_ 1 

20 

15.27 

100.00 

43.48 

L  J 

0 

0.00 

0.00 

0.00 

1  1 

MAJOR 

0 

0.00 

0.00 

0.00 

L-  ------  -J 

3 

2.29 

100.00 

6.52 

L  J 

0 

0.00 

0.00 

0.00 

L  1 

LT  COL 

. . H 

------  --j 

0 

0.00 

0.00 

0.00 

- . --H 

0 

0.00 

0.00 

0.00 

h . H 

1 

0.76 

100.00 

12.50 

h . + 

TOTAL  77  46  8 

58.78  35.11  6.11 


TOTAL 

4 

3.05 


20 

15.27 


20 

15.27 


3 

2.29 


1 

0.76 


131 

100.00 


FREQUENCY  MISSING  =  3 


APPENDIX  B 


TASK  CATEGORY  INSTRUMENT 


Name: 


Rank: 

MOS: 


MOS  TASK  CATEGORY  QUESTIONNAIRE 


Listed  below  are  98  task  categories.  For  each  task  category,  we  would  like  you  to 
make  the  following  four  ratings: 


1.  Is  the  task  a  PART  OF  THE  JOB  for  E3  and  E4  soldiers  in  the  MOS  you  are 

rating?  ^ 

=  No  (skip  ratings  2  -  4  and  go  to  the  next  task) 

_Y.  =  Yes  (continue  with  ratings  2-4) 


2.  How  IhffORTANT  or  critical  is  the  task  for  successful  performance  for  E3  and 
E4  soldiers  in  the  MOS? 

_5.  =  Extremely  high  importance  for  E3  and  E4  soldiers.  The  task  is  very 
central  to  the  job.  (Only  the  most  critical  tasks  should  get  this 
rating.)  ® 

_4_  =  High  importance.  The  task  is  central  to  the  job,  but  it  is  not  one 
of  the  most  critical  tasks. 

=  Moderate  importance.  The  task  is  part  of  the  job,  but  it  is  not 
central  to  the  job. 

—2.  ~  Low  importance.  The  task  is  part  of  the  job,  but  there  are  many 
tasks  that  are  more  important.  ^ 

_L  =  Very  minor  importance.  The  task  is  part  of  the  job,  but  it  is  not 
at  all  important  for  successful  job  performance. 


3.  How  FREQUENTLY  is  the  task  performed  by  E3  and  E4  soldiers  in  the  MOS? 

_2_  =  Very  frequently.  The  task  is  performed  much  more  frequently  than 

most  other  tasks. 

_2.  =  Frequently.  The  task  is  performed  about  as  often  as  other  tasks. 

_L  =  Occasionally  or  infrequently.  The  task  is  performed  much  less 
frequently  than  most  other  tasks. 
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How  DIFFICULT  and  demanding  is  it  for  E3  and  E4  soldiers  in  the  MOS  to 
perform  this  task  well? 

_2_  =  Very  difficult.  It  takes  considerable  skill  and  motivation  to  perform 
this  task  well.  Few  soldiers  can  perform  this  task. 

_2.  =  Moderately  difficult.  It  requires  some  skill  and  motivation  to 

perform  this  task  well,  but  most  soldiers  are  eventually  able  to  learn 
the  task. 

_L  =  Routine.  The  task  is  relatively  easy  and  requires  little  skill. 

Almost  all  soldiers  learn  the  task  very  quickly  and  are  able  to 
perform  the  task  with  little  difficulty. 


1.  Maintenance 


A.  Mechanical  Systems  Maintenance 

1-  Pcrfonn  operator  maintenance  checks  and  sftfvirf«  — 
follow  directions  in  Operator’s  Manual;  conduct  before, 
during,  after,  and  weekly  operator  checks  and  services 
on  vehicles,  trailers,  generators,  construction  equip* 
ment,  or  other  kinds  of  mechanical  apparatus. 

2.  Perform  operator  checks  and  services  on  weapons  -- 
check,  disassemble,  assemble,  clean,  lubricate,  and 
adjust  weapons,  including  pistols,  rifles,  machineguns, 
hand  grenades,  and  breechblocks. 

3.  Inspect  mechanical  systems  -  measure,  use  test  equip¬ 
ment  and  manuals,  and  observe  mechanical  equipment 
(e.g.,  engines,  transmissions,  brakes,  hydraulics, 
refrigeration  systems)  to  detect  problems  and  malfunc¬ 
tions. 

4.  Repair  weapons  -  after  the  cause  of  a  problem  in  a 
weapon  has  been  found,  fix  it  using  the  appropriate 
tools  and  necessary  replacement  parts  by  following 
directions  in  the  weapon’s  technical  manual. 

5.  Repair  mechanical  systems  -  after  the  cause  of  a 
problem  in  a  mechanical  part  has  been  found,  fix  it 
using  the  appropriate  tools  (e.g.,  wrenches,  screw¬ 
drivers,  gauges,  hammers,  soldering  equipment)  and 
necessary  replacement  parts  by  following  directions  in 
the  equipment’s  technical  manual. 
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6.  Troubleshoot  mechanical  systems  ~  find  the  cause  of 
malfunctions  in  mechanical  parts  and  equipment  using 
technical  manuals,  tools,  and  test  equipment  (e.g., 
calipers,  gauges,  torque  wrenches,  pressure  gauges). 

7.  Troubleshoot  weapons  —  find  the  cause  of  malfunctions 
in  weapons  using  technical  manuals,  tools,  and  test 
equipment. 

B.  Electrical  and  Electronic  Interns  Maintenance 

S.  Install  electronic  components  -  connect  electronic  and 
communications  equipment  (e.g.,  radios,  antennas, 
telephones,  teletypewriters,  radar,  power  supplies)  and 
check  system  for  operation. 

faspect  electrical  systems  -  measure,  use  test  equip¬ 
ment  and  manuals,  and  observe  electrical  systems  (e.g., 
generators,  wiring  harnesses,  switches,  relays,  circuit 
breakers)  to  detect  problems  and  malfunctions. 

10-  Inspect  electronic  systems  -  measure,  use  test  equip¬ 
ment  and  manuals,  and  observe  electronic  systems  (e.g., 
communications  equipment,  radar,  missile  and  tank 
computer  ballistics)  to  detect  problems  and  malfunc¬ 
tions. 

11-  Repair  electrical  systems  -  after  thp.  nf  an 
electrical  problem  has  been  found,  fix  it  with  the  ap¬ 
propriate  tools  (e.g.,  wire  strippers,  pliers,  soldering 
irons)  and  necessary  replacement  parts  by  following 
directions  in  the  equipment’s  technical  manual. 

B4 


Difficulty 


12.  Repair  electronic  components  —  After  the  cause  of  an 
electronics  problem  has  been  identified,  fix  it  with 
appropriate  tools  (e.g,,  tests  sets,  screwdrivers,  pliers, 
soldering  guns)  and  necessaiy  replacement  parts  by 
following  directions  in  the  equipment’s  technical 
manual. 

13.  Troubleshoot  electrical  systems  ~  find  the  cause  of 
malfunctions  in  electrical  parts  and  equipment  using 
technical  manuals,  tools,  and  test  equipment  (e.g., 
multimeters,  test  lights). 

14.  Troubleshoot  electronic  components  -  find  the  cause 

of  malfunctions  in  electronic  equipment  and  components 
using  technical  manuals,  tools  and  test  equipment  (e.g., 
oscilloscopes,  specialized  test  sets). 

n.  GENERAL  OPERATIONS 
C.  Pack  and  Load 

15.  Padc  and  load  materials  -  load  and  lash  materials  onto 
transport  vehicles  (land,  sea,  or  air)  to  secure  and 
protect  from  damage  or  loss  during  shipment. 

16.  Prepare  parachutes  —  inspect  cargo  and  personnel 
parachutes,  repair  or  replace  faulty  components,  and 
pack  parachutes  for  air  drop. 

17.  Prepare  equipment  and  supplies  for  air  drop  —  build  or 
assemble  platforms,  cushions,  and  riggings  for  para¬ 
chuting  supplies,  equipment,  and  vehicles. 
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Difficulty 


D.  Vehicle  and  Equipment  Operations 


18.  Operate  power  excavating  equipment  —  use  air  ham¬ 
mers  and  drills,  paving  breakers,  grinders,  backfill 
tampers,  or  other  hand  operated  power  equipment  in 
building  concrete,  stone,  or  other  structures  (e.g., 
roads,  fortifications,  buildings). 

19.  Operate  wheeled  vehicles  ~  drive  wheeled  vehicles 
over  roads  and  cross-country  in  response  to  mission, 
terrain,  and  traffic  regulations. 

20.  Operate  trade  vehides  -  drive  track  vehicles  (e.g., 
tank,  APC,  BFV)  in  response  to  mission,  terrain,  and 
traffic  controls. 

21.  Operate  boats  —  drive  boats  and  rafts. 

22.  Operate  lifting,  loading,  and  grading  equipment  — 
operate  fork  lifts,  cranes,  back-hoes,  graders,  and 
other  heavy  equipment  to  load,  unload,  or  move  heavy 
equipment,  supplies,  construction  materials  (e.g.,  culvert 
pipe,  building  and  bridge  parts),  or  terrain  (e.g.,  earth, 
rocks,  trees). 

K  Constmct/Assemble 

23.  Paint  —  prepare  surfaces  (clean,  remove  old  paint, 
sand)  and  apply  paint  with  brush,  roller,  or  spray. 

24.  Install  wire  and  cables  -  string  or  lay,  and  connect 
electrical  wire  or  communications  cables. 
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25.  Repair  plastic  and  fiberglass  --  fix  plastic  or  fiberglass 
parts  and  structures  by  cutting,  sawing,  drilling,  sand¬ 
ing,  filling,  gluing,  and  painting. 

26.  Repair  metal  ~  fix  metal  structures  or  parts  by  bend¬ 
ing,  cutting,  drilling,  welding,  hammering,  grinding, 
soldering,  and  painting. 

27.  Assemble  steel  structures  ~  erect  bridges,  antennas, 
and  other  steel  structures.  May  require  the  assistance 
of  others  and  use  of  heavy  equipment. 

28.  Install  pipe  assemblies  ~  place,  connect,  and  test  pipe 
assemblies  and  fixtures  (e.g.,  plumbing,  POL  pipelines 
and  pumps). 

29.  Construct  wooden  bufldings  and  other  structures  — 
measure,  saw,  nail,  plane  to  frame,  sheath,  and  roof 
buildings,  or  erect  trestles,  bridges,  and  piers. 

30.  Construct  masonry  buildings  and  structures  —  measure, 
lay  brick  or  concrete  blocks,  or  build  forms  and  pour 
concrete  to  construct  walls,  colunms,  field  fortifica¬ 
tions,  and  other  concrete  or  masonry  structures. 

F.  Technical  Procedures 

31.  Operate  gas  and  electric  powered  equipment  -  operate 
electric  generators,  air  compressors,  smoke  generators, 
quarry  machines,  mobile  washing  machines,  water 
pumps,  etc.,  to  produce  power  or  process  materials. 
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32.  Select  layout  and  dean  medical  or  dental  ggiripniffiit 
and  siippTif*^  —  prepare  treatment  areas  for  use  by 
following  prescribed  procedures  for  laying-out  instru¬ 
ments  and  equipment;  clean  equipment  and  area  for 
future  use. 

33.  Use  audiovisual  equipment  -  use  cameras  and 
videotape  to  record  sights  and  sounds  for  intelligence 
analysis,  training,  or  documentation. 

34.  Reproduce  printed  material  ~  operate  duplicating 
machines,  offset  presses,  and  similar  equipment  to 
reproduce  printed  materials;  collate  and  bind  materials 
using  various  types  of  bindery  equipment. 

35.  Operate  electronic  equipment  -  set  and  adjust  the 
controls  to  operate  electronic  equipment  (e.g.,  radio, 
computer  hardware,  missile  ballistics  controls). 

36.  Operate  radar  -  operate  radar  equipment  and  interpret 
signals. 

37.  Operate  computer  hardware  ~  operate  computer  hard¬ 
ware  such  as  tape  and  disk  drives,  optical  scanners, 
terminals,  and  other  input/output  devices. 

38.  Cook  ~  prepare  food  and  beverages  according  to 
recipes  and  meal  plans  (measure,  mix,  bake);  inspect 
fresh  food  and  staples  for  freshness;  clean  equipment 
and  work  area. 
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39.  Perfonn  medical  laboratoiy  procedures  -  conduct 
various  types  of  blood  tests,  urinalysis,  cultures,  etc. 

40.  Conduct  land  surveys  --  survey  terrain  to  determine 
elevations,  azimuths,  and  distances  of  terrain  features; 
record  information. 

41.  Provide  medical  or  dental  treatment  ~  give  medical 
attention  to  soldiers  in  the  field  or  in  medical  or 
dental  clinics,  or  give  veterinary  treatment  to  animals 
(e.g.,  administer  injections,  take  blood  pressure,  change 
sterile  dressings);  does  not  include  first  aid. 

G.  Make  Tedbnical  Drawings 

42.  Sketdi  maps,  overlays,  or  range  cards  -  use  standard 
symbols  to  make  sketches  of  terrain,  including  loca¬ 
tions  of  buildings  and  other  objects,  targets,  avenues 
of  approach,  and  maneuver  areas. 

43.  Produce  technical  drawings  ~  use  drafting  and  drawing 
equipment  to  make  technical  drawings  and  blueprints. 

44.  Draw  maps  and  overlays  ~  use  drafting,  graphics,  and 
related  techniques  to  draw  and  revise  maps  from  aerial 
photographs. 

45.  Draw  illustrations  -  using  pen,  pencil,  paint,  or  other 
media,  make  free  hand  drawings  and  illustrations. 
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m.  ADMINISTRATIVE 


HLOeiical 

46.  Type  --  type  information  using  a  typewriter,  teletype¬ 
writer,  keypunch,  or  computer  terminal. 

47.  Prepare  technical  forms  and  document  -  follow 
standardized  procedures  to  prepare  or  complete  forms 
and  documents  (e.g.,  personnel  records,  legal  brie&). 

48.  Record,  ffle.  and  dispatch  information  -  collect,  copy, 
update,  sort,  index,  file,  and  retrieve  information  (e.g., 
mail,  training  rosters,  personnel  statistics,  supply 
inventories). 

49.  Receive,  store,  and  issue  supplies,  equipment  and  other 
materials  ~  inspect  materials  and  review  paperwork 
when  receiving  materials;  sort,  transport,  and  store 
materials;  issue  or  ship  material  to  authorized  person¬ 
nel  or  units. 

I.  Communication 

^0.  Use  hand  and  arm  signals  ~  communicate  messages  and 
instructions  using  hand  and  arm  signals. 

51-  Read  technical  manuals,  field  manuals,  rft|Tiilatinn!e„  ^nrf 
other  publications  ~  use  index  and  table  of  contents 
to  find  location  of  needed  information;  locate  informa¬ 
tion;  read  instructions,  diagrams,  charts,  and  tables. 

52.  Use  maps  -  read  and  interpret  map  symbols  and 

identify  terrain  features  in  order  to  orient  map  to  your 
position  in  the  field;  determine  grid  coordinates; 
determine  directions;  identify  roads,  towns,  etc. 
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53.  Send  and  receive  radio  messages  -  use  standardized 
radio  codes  and  procedures  to  transmit  and  receive 
messages  and  other  information. 

54.  Give  oral  reports  ~  use  standard  communication 
procedures  to  organize  information  (e.g.,  SALUTE, 
challenge  and  password,  call  for  fire). 

55.  Receive  clients,  patients,  guests  -  schedule,  greet,  and 
give  routine  information  to  persons  seeking  medical, 
dental,  legal,  or  counseling  services. 

56.  Give  directions  and  instructions  --  verbally  give  infor¬ 
mation,  instructions,  or  directions  to  others. 

57.  Write  documents  and  correspondence  -  draft  letters, 
reports,  memos,  etc.;  proofread  and  edit 

58.  Write  and  deliver  presentations  -  prepare  scripts  for 
formal  presentations,  including  radio  and  television 
broadcasts  and  briefings. 

59.  Interview  ~  gather  information  from  clients,  patients, 
witnesses,  prisoners,  or  other  persons. 

60.  Provide  counseling  and  other  interpersonal  mteivftn- 
lipns  ~  conduct  personal  adjustment  counseling  with 
individuals  and  groups;  use  interpersonal  relations  skills 
to  solve  relationship  problems. 
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J.  Analyze  Infonnation 


61.  Decode  data  ~  use  coding  systems  and  rules  to 
decipher  and  interpret  coded  information  (e.g.,  use 
CEOI,  interpret  symbols/signs). 

62.  Analyze  electronic  signals  ~  analyze  electronics  signals 
to  detect  threat  transmitters  and  electronic  counter¬ 
measures. 

63.  Analyze  weather  condition.s  —  determine  weather 
conditions  and  analyze  their  effects  on  tactical 
operators. 

64.  Order  equipment  and  supplies  —  determine  needs  and  • 
requisition  needed  supplies,  materials,  and  equipment 

65.  Estimate  time  and  cost  of  maintenance  opcTatinns  — 
estimate  equipment  downtime  and  cost  of  repairs, 
including  parts  and  labor. 

66.  Plan  placement  or  use  of  tactical  equipment  ~  using 
maps  and  on-site  inspection,  identity  positions  and 
areas  to  be  used  for  cover  and  concealment  and  to 
place  weapons,  fortifications,  mines,  and  detectors. 

67.  Translate  foreign  languages  ~  translate  written  or 
spoken  foreign  language  communications. 

68.  Analyze  intelligence  data  ~  determine  importance  and 
reliability  of  information;  use  information  to  determine 
identity,  capabilities,  disposition,  and  movement  of 
enemy  forces. 
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K.  Applied  Math  and  Data  Processing 
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69.  Control  money  --  keep  accounting  records;  disperse  and 
collect  money  and  money  orders. 

70.  Determine  firing  data  for  indirect  fire  weapons  -- 
using  maps,  firing  charts,  and  targeting  and  ballistics 
information,  determine  elevation  and  azimuth  needed 
for  engaging  targets. 

71.  Compute  statistics  or  other  mathematical  calmlarinm 
~  select  formulas  and  make  mathematical  calculations, 
with  or  without  using  calculators  or  computers;  report 
results. 

72.  Provide  programming  and  data  processing  support  for 
computer  operations  -analyze  data  processing  needs; 
select  or  prepare,  edit,  test,  and  run  computer 
programs;  document  process  and  results. 

L.  Control  Air  Traffic 

73.  Control  air  traffic  —  coordinate  departing,  en  route, 
arriving,  and  holding  aircraft  by  monitoring  radar 
equipment,  communicating  with  aircraft  and  other  air 
traffic  control  units. 

IV.  COMBAT 

M.  Individual  Combat 

74.  Use  hand  grenades  -  identify,  inspect,  arm,  throw, 
and  secure  hand  grenades. 
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75.  Protect  against  NRC  hazards  —  use  protective  dothing, 
masks,  and  decontamination  equipment  to  protect  self, 
others,  equipment,  and  supplies  from  nuclear, 
biological,  and  chemical  hazards. 

76.  Handle  demolitions  or  mines  --  store,  place,  charge, 
discharge,  and  disarm  explosives,  demolition  devices,  or 
mines. 

77.  Engage  in  hand-to-hand  combat  -  use  offensive  and 
defensive  maneuvers  to  overcome  hostile  individuals. 

78.  Fire  individual  weapons  ~  aim,  track  and  fire  in¬ 
dividual  weapons,  such  as  rifles,  pistols,  machineguns, 
and  LAW  at  designated  targets;  load,  reduce  a  stop¬ 
page,  and  clear  weapons. 

79.  Control  individuals  and  crowds  —  apprehend  and  search 
suspected  criminals  or  enemy  soldiers;  guard  prisoners; 
participate  in  riot  control;  perform  guard  duty. 

SO.  Customs  and  laws  of  war  -  use  knowledge  of  Geneva 
convention  and  military  SOP  concerning  treatment  of 
enemy  personnel,  engagement  of  the  enemy,  conduct  of 
military  protocol  and  ceremony,  guard  duty,  and 
physical  readiness. 

81.  Navigate  ~  during  the  day  or  night,  with  or  without  a 
map,  locate  positions  and  move  from  point  to  point  in 
response  to  terrain  features  (e.g.,  for  cover  or  con¬ 
cealment),  battle  conditions,  and  mission. 

B14 


A 

0 

0) 

b 

o 

>1 

p 

o 

Id 

p 

0 

0) 

u 

p 

4J 

0 

& 

U 

04 

o 

Id 

s 

u 

P4 

H 

82.  Survive  in  the  field  —  react  to  direct  or  indirect  fire; 
prepare  individual  battle  positions;  camouflage  self  and 
equipment;  observe  security  procedures. 

N.  Crew-served  Weapons 

83.  Load  and  unload  field  artillery  or  tank  pins  —  operate 
breech  controls  and  handle  ammunition  (stow  and  load) 
to  prepare  guns  for  firing;  unload  or  extract  unused 
rounds  or  misfires. 

84.  Fire  heavy  direct  fire  weapons  (e.g.,  tank  main  guns, 
TOW  missile,  BFV  cannon)  —  using  weapon  sights, 
manipulate  weapon  controls  to  aim,  track,  and  fire  on 
targets. 

85.  Prepare  heavy  weapons  for  tactiral  use  ~  position  and 
prepare  for  firing  heavy  tactical  weapons,  such  as 
missiles,  field  artillery,  and  anti-aircraft  systems. 

86.  Place  and  camouflage  tactical  equipment  and  matftriak 
in  the  field  ~  place  mines,  detectors,  chemicals,  and 
camouflage  materials  into  position  in  the  battlefield. 

87.  Fire  indirect  fire  weapons  (e.g.,  field  artflleiy)  --  lay 
weapon  by  adjusting  azimuth  and  elevation  controls  in 
response  to  fire  commands. 

O.  Give  First  Aid 

88.  Give  first  aid  ~  carry  out  first  aid  procedures  (e.g., 

CPR,  put  on  field  dressing,  prevent  shock). 
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Difficulty 


P.  Identify  Targets 


89.  Detect  and  identify  targets  -  with  or  without  optical 
devices  (e.g.,  night  sights,  weapon  sights,  binoculars), 
locate  possible  targets,  and  identify  type  (e.g.,  troops, 
tanks,  aircraft)  and  nomenclature. 

IV.  Supervision 

90.  Plan,  organise,  monitor  -  assign  work  tasks,  supervise 
performance  of  tasks,  conduct  inspections,  and  monitor 
equipment  condition  and  supplies. 

9 1.  Oarify  roles,  provide  feedback  -  monitor  performance 
and  counsel  subordinates. 

92.  Provide  infnrmarinn  ■  pass  on  information  concerning 
mission  and  requirements  to  subordinates. 

93.  Recognize,  reward .  provide  formal  and  informal 
rewards  and  recognition  for  good  performance;  recom¬ 
mend  soldiers  for  promotion  or  awards. 

94.  Support  -  listen  to  subordinates’  personal  problems; 
counsel,  assist,  or  arrange  assistance,  as  appropriate. 

95.  Train,  develop  -  plan  and  conduct  individual  and  team 
training,  provide  career  counseling,  and  provide  oppor¬ 
tunities  for  leadership. 

96.  Discipline,  punish  -  provide  formal  or  informal  discipli¬ 
nary  measures  to  subordinates. 
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97.  Act  as  model  -  set  the  example  for  subordinates. 

98.  Conduct  tactical  operations  -  supervise  and  direct 
activities  of  the  unit  during  tactical  operations. 
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Name 


TASK  CATEGORY  RATING  EXERCISE 
EVALUATION  SHEET 


Task  Ca'tegoxy  Definitions 


1.  How  clear  were  the  task  category  definitions? 

5  6  7 

Very  clear 

used  in  the  task  category 

5  6  7 

There  were 
no  terms 
I  did  not 
understand 

3.  Which  task  category  was  most  difficult  to  judge?  _ 

4.  Which  task  category  was  least  difficult  to  judge?  _ 

5 .  What  percentage  of  your  j  ob  was  covered  by  these  task  categories? 

0%  10%  20%  30%  40%  50%  60%  70%  80%  90%  100% 

If  you  answered  less  than  100%,  what  task  categories  should  be  added? 


Not  at  all 
clear 


Neither  clear 
nor  unclear 


2.  How  well  did  you  understand  the  terms 
definitions? 


There  were 
many  terms 
I  did  not 
understand 


There  were 
some  terms 
I  did  not 
understand 
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Task  Category  Ratings 


1.  How  clear  were  the  instructions  for  making  the  ratings? 

1  2  3  4  5  6  7 

Not  at  all  Neither  clear  Very  clear 

clear  nor  unclear 

2.  Were  the  ratings  easy  or  difficult  to  make? 

1  2  3  4  5  6  7 

Very  Neither  easy  Very 

®asy  nor  difficult  difficult 


3 .  How  confident  did  you  feel  about  the  accuracy  of  your  ratings? 
1  2  3  4  5  6  7 


Not  at  all  Somewhat 

confident  confident 


Very 

confident 


General  Evaluation 

1.  Which  of  the  following  would  you  change  (check  all  that  apply): 
_  Task  Category  Definitions 

_  Task  Category  Rating  Instructions  and  Procedures 


2.  What  changes  would  you  make? 


MATCHING  EXERCISE  INSTRUCTIONS: 
MOS  TASKS  AND  TASK  CATEGORIES 


In  this  portion  of  the  workshop,  we  are  interested  in  finding 
out  how  well  our  list  of  98  task  categories  can  be  matched  up 
with  specific  tasks  from  the  Soldier's  Manual  (SM)  and  from  the 
Army  Occupational  Survey  Program  (AOSP) . 

We  have  reviewed  the  SM  and  the  AOSP  and  have  selected  30 
tasks  that  cover  the  range  of  responsibilities  for  Skill  Level  1 
and  Skill  Level  2  soldiers.  Following  each  task,  there  are  three 
blank  lines.  In  those  blanks,  we  want  you  to  list  the  numbers  of 
up  to  3  task  categories  that  best  describe  that  task.  You  can 
find  the  names  and  numbers  of  the  task  categories  on  the  "MOS 
Task  Category  Questionnaire"  that  you  completed  during  the  previous 
exercise.  This  form  provides  the  number,  the  name,  and  a  brief 
description  of  each  of  the  98  task  categories.  Or,  if  you  would 
prefer,  the  task  category  names  and  numbers  also  can  be  obtained 
from  the  attached  handout,  "List  of  Task  Categories"  (which  does 
not  include  any  descriptions  of  the  categories) . 

As  you  review  the  task  categories,  first  find  the  task 
category  that  best  describes  the  task.  Put  the  number  of  this 
task  category  in  the  first  column,  under  "1st  Category/Activity. " 
Then  find  the  niimber  of  the  task  category  that  provides  the 
second  best  description  of  the  task  and  mark  this  number  under 
"2nd  Category/ Activity."  Finally,  do  the  same  for  the  third  task 
category. 

You  do  not  have  to  fill  in  all  three  blanks  for  all  of  the 
tasks.  You  may  find  that  only  one  or  two  categories  describe 
some  tasks.  Feel  free  to  leave  the  remaining  column (s)  blank. 

find  that  none  of  the  task  categories  describes  a  task, 
simply  write  "NONE"  across  the  three  blanks. 

Also,  you  are  not  limited  in  the  number  of  times  that  you 
can  use  one  of  the  task  categories.  For  example,  if  the  same 
task  category  describes  all  of  the  tasks  on  the  Task  List,  you 
can  use  that  category  for  every  task. 
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List  of  Task  Activities 


1*  Perform  operator  maintenance  checks  and  services 

2.  Perform  operator  checks  and  services  on  weapons 

3.  Inspect  mechanical  systems 

4 .  Repair  weapons 

5.  Repair  mechanical  systems 

6.  Troiableshoot  mechanical  systems 

7 .  Troubleshoot  weapons 

8.  Install  electronic  components 

9.  Inspect  electrical  systems 

10.  Inspect  electronic  systems 

11.  Repair  electrical  systems 

12.  Repair  electronic  components 

13 .  Troubleshoot  electrical  systems 

14.  Troubleshoot  electronic  components 

15.  Pack  and  load  materials 

16 .  Prepare  parachutes 

17 .  Prepare  equipment  and  supplies  for  air  drop 

18.  Operate  power  excavating  equipment 

19.  Operate  wheeled  vehicles 

20.  Operate  track  vehicles 

21.  Operate  boats 

22.  Operate  lifting,  loading,  and  grading  equipment 

23.  Paint 

24.  Install  wire  and  cables 

25.  Repair  plastic  and  fiberglass 

26.  Repair  metal 

27 .  Assemble  steel  structures 

28.  Install  pipe  assemblies 

29 .  Construct  wooden  buildings  and  other  structures 

30.  Construct  masonry  buildings  and  structures 

31.  Operate  gas  and  electric  powered  equipment 

32.  Select,  layout  and  clean  medical  or  dental  equipment 

33.  Use  audiovisual  equipment 

34.  Reproduce  printed  material 

35.  Operate  electronic  equipment 

36.  Operate  radar 

37 .  Operate  computer  hardware 

38.  Cook 

39 •  Perform  medical  laboratory  procedures 

40.  Conduct  land  surveys 

41.  Provide  medical  or  dental  treatment 

42.  Sketch  maps,  overlays,  or  range  cards 

43.  Produce  technical  drawings 

44.  Draw  maps  and  overlays 

45.  Draw  illustrations 

46.  Type 

47 .  Prepare  technical  forms  and  documents 

48.  Record,  file,  and  dispatch  information 

49.  Receive,  store,  and  issue  supplies,  equipment,  and  other 
materials 
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50.  Use  hand  and  arm  signals 

51.  Read  technical  manuals,  field  manuals,  regulations,  and 
other  publications 

52 .  Use  maps 

53 .  Send  and  receive  radio  messages 

54.  Give  oral  reports 

55.  Receive  clients,  patients,  guests 

56.  Give  directions  and  instructions 

57 .  Write  docximents  and  correspondence 

58.  Write  and  deliver  presentations 

59 .  Interview 

60.  Provide  counseling  and  other  interpersonal  interventions 

61.  Decode  data 

62.  Analyze  electronic  signals 

63 .  Analyze  weather  conditions 

64 .  Order  equipment  and  supplies 

65.  Estimate  time  and  cost  of  maintenance  operations 

66.  Plan  placement  or  use  of  tactical  equipment 

67.  Translate  foreign  languages 

68.  Analyze  intelligence  data 

69 .  Control  money 

70.  Determine  firing  data  for  indirect  fire  weapons 

71.  Compute  statistics  or  other  mathematical  calculations 

72.  Provide  prograioming  and  data  processing  support  for  computer 
operations 

73.  Control  air  traffic 

74.  Use  hand  grenades 

75.  Protect  against  NBC  hazards 

76.  Handle  demolitions  or  mines 

77 .  Engage  in  hand-to-hand  combat 

78.  Fire  individual  weapons 

79.  Control  individuals  and  crowds 

80.  Customs  and  laws  of  war 

81.  Navigate 

82.  Survive  in  the  field 

83.  Load  and  unload  field  artillery  or  tank  guns 

84.  Fire  heavy  direct  fire  weapons  (e.g.,  tank  main  guns,  TOW 
missile,  IFV  cannon) 

85.  Prepare  heavy  weapons  for  tactical  use 

86.  Place  and  camouflage  tactical  equipment  and  materials  in  the 
field 

87.  Fire  indirect  fire  weapons  (e.g.,  field  artillery) 

88.  Give  first  aid 

89.  Detect  and  identify  targets 

90.  Plan,  organize,  monitor 

91.  Clarify  roles,  provide  feedback 

92 .  Provide  information 

93.  Recognize,  reward 

94 .  Support 

95.  Train,  develop 

96.  Discipline,  punish 

97 .  Act  as  model 

98.  Conduct  tactical  operations 
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Task 

No,  Task  Name 


1st  2nd  3rd 

Task  Task  Task 

Category/  Category/  Category/ 
Activity  Activity  Activity 


24.  Put  on  and  wear  protective 
clothing  in  accordance  with 
established  (MOPP)  levels 

25.  Put  on  a  field  or  pressure 
dressing 

26.  Perform  expedient  repairs 

27.  Troubleshoot  engine  cooling 
system 

28.  Troubleshoot  engines 

29.  Troubleshoot  steering 
system 

30.  Slave  start  disabled  vehicle 
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NAME 


TASK  MATCHING  EXERCISE 

Task  List  for  71L 
Form  A 


1st  2nd 

Task  Task 

Category/  Category/ 

[Oj.  Task  Name  Activity  Activity 

1.  Type  a  basic  comment  to  a 
disposition  form 

2.  Type  a  military  letter  _ 


3.  Type  a  joint  messageform 
(DD  Form  173/1) 

4.  Prepare  a  requisition  for 
publications  and/or  blank 
forms — AUTODIN  (DA  Form 
4569) 

5.  Type  military  orders 


6.  Type  a  memorandum 


7 .  Dispatch  outgoing  dis¬ 
tribution 

8.  Establish  functional  files 


9.  File  documents/correspon¬ 
dence 

10.  Type  a  second  or  subsequent 
comment  to  a  disposition 
form 

11.  Type  straight  copy  material 


12 .  Assemble  correspondence 


13.  Receipt/transfer  classified 
material 


3rd 

Task 

Category/ 

Activity 
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Task 

No. 

14. 

15. 

16. 

17. 

18. 

19. 

20. 

21. 

22. 

23. 

24. 

25. 

26. 


Task  Name 


1st  2nd 

Task  Task 

Category/  Category/ 
Activity  Activity 


Safeguard  "FOR  OFFICIAL  USE 
ONLY"  (FOUO)  material 

Perform  operator  main¬ 
tenance  on  an  M16A1  rifle, 
magazine,  and  ammunition 

Load,  reduce  a  stoppage, 
and  clear  an  M16A1  rifle 

Camouflage  yourself  and 
your  individual  equipment 

Practice  noise,  light,  and 
litter  discipline 

Determine  the  grid  coor¬ 
dinates  of  a  point  on  a 
military  map  using  the 
military  grid  reference 
system 

Determine  a  magnetic 
azimuth  using  a  compass 

Put  on  a  field  or  pressure 
dressing 

Maintain  an  M17  series 
protective  mask 

Put  on  and  wear  an  M17 
series  protective  mask  with 
hood 

Put  on  and  wear  protective 
clothing  in  accordance  with 
established  mission- 
oriented  protective  posture 
(MOPP)  levels 

Know  your  rights  and 
obligations  as  a  prisoner 
of  war 

Administer  nerve  agent  an¬ 
tidote  to  self  (self-aid) 


3rd 

Task 

Category/ 

Activity 
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Task 

No.  Task  Name 


1st  2nd 

Task  Task 

Category/  Category/ 
Activity  Activirv 


27.  Receive,  maintain,  and  con¬ 
trol  office  equipment 

28.  Control  expendable/non-ex- 
pendable  supplies 


3rd 

Task 

Category/ 

Activity 
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APPENDIX  D 


ATTRIBUTE  INSTRUMENT 


ATTRIBUTE  DEFINITIONS 


D1  ' 


This  is  the  ability  to  use  and  understand  spoken  and  written  lanquaqe 
and  to  commumcate  with  others.  It  involves  "catching  on"  to  what’s  hap¬ 
pening  and  coming  up  with  and  understanding  words  and  ideas. 

HIGH  on  VERBAL  ABILITY  understands  the  meaning  of  even 
difficult  or  unusual  words;  understands  written  material  quickly;  rapidly 
comes  up  with  words  or  ideas;  quickly  identifies  the  relationship  between 
words  or  ideas;  and,  clearly  communicates  ideas  and  information. 


An  individual  who  has  AVERAGE  VERBAL  ABILITY  understands  the  meaninq 
of  most  common  words  but  does  not  know  very  difficult  words;  understands 
material  in  a  reasonable  amount  of  time,  but  may  not  pick  up 
subtle  Ideas  or  complex  directions;  can  come  up  with  appropriate  words  or 
Ideas  in  reasonable  time;  can  identify  obvious  relationships  between  words 
or  ideas  quickly  but  requires  more  time  to  grasp  complicated  or  unusual 
relationships;  and,  communicates  ideas  and  information  moderately  well. 


An  individual  LOW  on  VERBAL  ABILITY  does  not  know  the  meaning  of  even 
common  words  or  phrases;  cannot  understand  most  written  material;  has  dif¬ 
ficulty  coming  up  with  appropriate  words  or  ideas;  cannot  identify  the 

5*nn  between  most  words  or  ideas;  and,  has  difficulty  communicat¬ 
ing  ideas  and  information. 


Reasoning 


is  the  ability  to  discover  a  rule  or  principle  underlying  the 
relationship  between  two  or  more  objects  and  apply  it  in  solving  a  problem. 

si  so  includes  the  ability  to  use  logic  in  drawing  conclusions  from 
available  information. 

An  individual  HIGH  on  REASONING  who  is  shown  a  set  of  objects  can 
quickly  identify  the  common  theme  or  characteristic  that  those  objects 
share,  and  can  tell  whether  new  objects  do  or  do  not  share  the  same  charac¬ 
teristic;  can  study  a  sequence  of  objects  or  figures  and  can  readily  deter¬ 
mine  what  changes  occur  from  the  first  to  the  second  object,  from  the 
second  to  the  third  object,  and  so  on,  and  can  quickly  tell  what  the  next 
object  in  the  series  would  look  like;  and,  when  given  a  set  of  facts  and  a 
set  of  conclusions,  can  quickly  determine  which  conclusions  are  correct. 


An  individual  with  AVERAGE  REASONING  who  is  shown  a  set  of  objects  may 
occasionally  have  trouble  identifying  the  common  theme  or  characteristic 
that  those  objects  share,  and  so  may  not  be  able  to  tell  whether  new  ob¬ 
jects  do  or  do  not  share  the  same  characteristic;  usually  can  determine 
what  changes  occur  from  one  object  to  the  next  in  a  series  of  objects,  but 
may  occasionally  have  trouble;  and  sometimes  may  have  difficulty  deciding 
what  the  next  object  in  the  series  would  look  like;  and,  when  given  a  set 
of  facts  and  a  set  of  conclusions,  may  have  difficulty  determining  which 
conclusions  are  correct. 


An  individual  LOW  on  REASONING  who  is  shown  a  set  of  objects  often  has 
a  great  deal  of  trouble  identifying  the  common  theme  or  characteristic  that 
those  objects  share,  and  so  often  will  not  be  able  to  tell  whether  new 
objects  do  or  do  not  share  the  same  characteristic;  has  trouble  determining 
what  changes  occur  from  one  object  to  the  next  in  a  series  of  objects,  and! 
therefore,  is  not  able  to  tell  what  the  next  object  in  the  series  would 

when  given  a  set  of  facts  and  a  set  of  conclusions,  cannot 
determine  which  conclusions  are  correct. 
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Number  Ability 


This  is  the  ability  to  perform  arithmetic  calculations  and  manioula- 
tions  quickly  and  correctly,  and  to  solve  mathematical  word  problems. 

performs  simple  computations 
Without  error  (i  e.,  addition,  subtraction,  multiplication,  and 
division);  can  use  formulas  to  quickly  and  accurately  solve  almost  all 

matica/Iord^problSmL^^"  quickly  and  correctly  solve  even  difficult  mathe- 

An  individual  who  has  AVERAGE  NUMBER  ABILITY  performs  simple  computa- 
tions  with  moderate  speed  and  few  errors  (i.e.,  addition,  subtraction, 
multiplication,  and  division);  can  use  formulas  to  solve  some  number  prob- 
laTLIi  time  with  minimal  errors;  and,  can  solve  simple  mathe- 

without  help  unable  to  solve  difficult  word  problems 

An  individual  LOW  on  NUMBER  ABILITY  has  difficulty  perforrainq  simole 
computations  (i.e.,  addition,  subtraction,  multiplication,  and  di?isS 
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•  j  ii^  locate  objects  that  are  partially  hidden,  or  to 

Identity  objects  that  have  been  turned  backwards  or  upside  down  or  have 
been  rotated  sideways.  It  includes  the  ability  to  mentally  break  a  figure 
into  pieces  and  come  up  with  a  new  arrangement,  and  the  ability  to  remember 
how  an  object  was  turned  or  configured  and  where  it  was  located. 

.1,  .  SPATIAL  ABILITY  can  quickly  find  a  simple  form 

that  has  been  hidden  inside  a  complex  pattern;  nearly  always  can  remember 
where  an  object  was  located  and  how  it  was  turned  or  configured;  can  quick- 

•  j  accurately  identify  a  path  through  a  complex  maze;  and,  always  can 
identify  exactly  how  a  complex  object  will  look  when  it  has  been  rearranged 
or  turned  upside  down,  backwards,  or  sideways. 

An  individual  with  AVERAGE  SPATIAL  ABILITY  can  find  a  simple  form  that 
has  been  hidden  inside  a  complex  pattern,  though  it  may  require  some  time; 
usually,  but  not  always,  can  remember  where  an  object  was  located  and  how 
It  was  turned  or  configured;  can  quickly  and  accurately  identify  a  path 
through  a  simple  maze,  but  will  require  a  great  deal  of  time  and  have  dif- 
nculty  identifying  a  path  through  a  complex  maze;  and,  usually  can  identi¬ 
fy  how  a  complex  object  will  look  when  it  has  been  rearranged  or  turned 
upside  down,  backwards,  or  sideways. 


An  individual  LOW  on  SPATIAL  ABILITY  has  great  difficulty  and  may  not 
be  able  to  find  a  simple  form  that  has  been  hidden  inside  a  complex  pat¬ 
tern;  usually,  but  not  always,  cannot  remember  where  an  object  was  located 
or  how  it  was  turned  or  configured;  has  great  difficulty  identifying  a  path 
through  even  a  simple  maze;  and,  cannot  identify  how  a  complex  object  will 
look  when  it  has  been  rearranged  or  turned  upside  down,  backwards,  or  side- 

W3ys  • 


Closure 


or  partlMnf5ma?iin!^ 

An  individual  HIGH  on  CLOSURE  is  able  to  read  letters  and  words  and 

recognize  objects  quickly,  even  if  a  large  portion  of  the  letter,  word  or 
object  has  been  hidden  or  erased.  wora,  or 

An  individual  with  AVERAGE  CLOSURE  is  able  to  read  most  letters  and 

objects  when  portions  of  the  letter,  word,  or 
ject  have  been  hidden  or  erased,  but  may  require  a  fair  amount  of  time  to 

nortiUn^hac^h^^  recognize  the  item  if  a  large 

portion  has  been  hidden  or  erased.  ^ 

An  Individual  LOW  on  CLOSURE  has  great  difficulty  recognizing  letters 
words,  and  objects  that  have  been  partially  hidden  or  erased;  and  inav 

a“;ro?  i?ase‘d'“’ 
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Mental  Information  Processing 

4.L  ability  to  react  quickly  to  a  stimulus.  It  also  involves 

the  ability  to  attend  carefully  to  information  that  is  being  presented 
(e.g.,  in  a  conversation,  on  a  computer  screen),  even  when  other  events  are 
occurring  and  competing  for  one’s  attention.  Finally,  it  involves  the 
ability  to  direct  attention  to  and  perform  two  tasks  at  the  same  time,  when 
required.  ’ 


The  individual  HIGH  on  MENTAL  INFORMATION  PROCESSING  is  able  to  react 
quickly  to  sounds  (e.g.,  bells,  snapping  twigs)  and  sights 
(lights,  unusual  shadows);  can  focus  attention  on  one  event  or  problem,  and 
can  block  out  distracting  noises,  sights,  or  events  that  are  competing  for 
attention;  when  required,  is  able  to  focus  attention  on  and  perform  two 
tasks  very  well  at  the  same  time  (e.g.,  tracking  a  blip  closely  on  radar 
while  doing  computations  on  a  calculator). 


The  individual  with  AVERAGE  MENTAL  INFORMATION  PROCESSING  ability  is 
•  quickly  to  sounds  (e.g.,  bells,  snapping  twigs)  and 

sights_  (lights,  unusual  shadows);  usually  can  focus  attention  on  one  event 
or  prouiGfn,  but  may  occasionally  havG  troubla  blocking  out  distracting 
noises,  sights,  or  events  that  are  competing  for  attention;  when  required, 
IS  able  to  focus  attention  on  and  perform  two  tasks  at  the  same  time,  but 
performance  on  one  or  both  of  the  tasks  may  be  rather  slow  or  inaccurate. 

The  individual  LOW  on  MENTAL  INFORMATION  PROCESSING  reacts  rather 

sounds  (6-9: >  bells,  snapping  twigs)  and  sights  (lights,  unusual 
shadows),  has  great  difficulty  focusing  attention  on  one  event  or  problem 
when  distracting  noises,  sights,  or  events  are  competing  for  attention;  has 
trouble  focusing  attention  on  and  performing  more  than  one  task  at  a  time, 
and  generally  will  ignore  one  task  completely  while  performing  the  other. 
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This  is  the  ability  to  notice  details  about  thin9S  (letters,  objects 
numbers,  symbols,  or  patterns)  quickly  and  correctly.  This  involves  rapid¬ 
ly  noting  changes  and/or  the  way  things  differ  or  are  alike. 


An  individual  HIGH  on  PERCEPTUAL  SPEED  AND  ACCURACY  is  able  to  see  and 
procep  things  quickly  and  accurately;  can  quickly  perform  complex  percep- 
tual .tasks  (e.g.,  compare  several  objects  and  eliminate  all  those  that  are 
not  identical);  and,  is  rarely  distracted  by  irrelevant  cues. 


An  individual  who  has  AVERAGE  PERCEPTUAL  SPEED  AND  ABILITY  can  see  and 
process  things  with  moderate  speed  and  few  mistakes;  can  perform  simple 
perceptual  tasks  well  (e.g.,  compare  two  objects  for  similarity),  but  has 
difficulty  with  more  complex  tasks;  and,  is  occasionally  distracted  by 
irrelevant  cues.  ^ 


An  individual  LOW  on  PERCEPTUAL  SPEED  AND  ACCURACY  takes  a  great  deal 
Of  time  to  see  and  process  things;  makes  many  errors  when  performing  simole 
perceptual  tasks;  and,  is  easily  distracted  by  irrelevant  cues. 
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Memory 


is  the  ability  to  recall  previously  learned  material.  It  in- 
cludes  the  ability  to  remember  miscellaneous  facts  and  trivia,  as  well  as 


An  individual  HIGH  on  MEMORY  is  able  to  accurately  recall  information 
tnn"*  books,  or  magazine  articles  with  little  or  no  prompt - 

Si,rp^  mLf  ®  "figural"  information,  such  as 

pictures,  maps,  faces,  and  scenes  from  places  the  individual  has  visited. 

An  individual  with  AVERAGE  MEMORY  is  able  to  recall  quite  a  bit  of 
information  from  conversations,  books,  or  magazine  articles,  but  may  re¬ 
quire  some  prompting  and  may  omit  some  details;  and,  is  able  to  recall  most 

figural  information,  but  may  have  trouble  recalling  certain  features  of 
maps,  pictures,  or  faces.  letuures  or 

An  individual  LOW  on  MEMORY  is  able  to  recall  some  information  from 
conversatpns,  books,  or  magazine  articles,  but  often  requires  a  great  deal 
of  prompting  and  omits  significant  details;  has  trouble  recalling  figural 

'•«»9n1ze  many  maps,  p1ctJ?L,  o? 
faces  that  have  been  encountered  previously. 
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1  ability  to  understand  mechanical,  shop,  automotive  and 

operate?*'  *"*^  knowledge,  and  the  way  machines,  tools,  and  equipment 

''3  high  on  MECHANICAL  COMPREHENSION  can  quickly  recognize 

IS  p?Sc???arsit‘  at?o*lf  ™Schan1carS?ISSS?s 

in  practical  situations  (e.g.,  how  levers  and  gears  work,  what  causes  hiah 
pressure  to  occur  in  the  flow  of  water);  is  able  to  learn  and  fiSSSS  out 

quickly  and  accurately;  and,  has^no  dif- 
ficulty  solving  practical  mechanical  or  electrical  problems. 

An  individual  with  AVERAGE  MECHANICAL  COMPREHENSION  skills  can  recoa- 

eleLnts  the  relationship  of  physical  forces  and  mechanical 

elements  in  practical  situations  (e.g.,  how  levers  and  gears  work)  with 
moderate  speed  and  accuracy;  is  able  to  learn  and  figuri  out  meShan"Sal 

errors;  and,  has  only  a  small 

amount  of  difficulty  solving  practical  mechanical  or  electrical  problems. 

.nH  /J  '"^''1'^“?'  kow  on  MECHANICAL  COMPREHENSION  is  unable  to  recognize 

IS  mechanical  ?ll«"s 

hSs^SSSSt  rflfivSir""*!  “"iierstand  mechanical  terms;  and, 

has  great  difficulty  solving  practical  mechanical  or  electrical  problems. 


la  n  ability  to  make  coordinated  movements  of  any  two  limbs 

(e.g.,  both  hands,  both  feet,  or  one  foot  and  one  hand)  with  high  precision 
and  speed.  Eye-limb  coordination  is  most  important  for  tasks  where  the 
body  IS  seated  or  standing  while  two  or  more  limbs  are  in  motion.  For  ex¬ 
ample,  eye-limb  coordination  is  very  important  for  flying  an  airplane, 
because  the  pilot  must  make  very  precise,  coordinated  hand  and  ^move¬ 
ments  to  maintain  the  desired  altitude  and  heading.  Eye-limb  coordination 
does  not  apply  to  tasks  in  which  the  movement  of  other  parts  of  the  body 
must  be  integrated  with  arm  or  leg  movements,  such  as  hitting  a  baseball. 

An  individual  HIGH  on  EYE-LIMB  COORDINATION  is  very  good  at  doing 
tasks  that  require  very  precise,  coordinated  movements  of  two  arms,  two 
legs,  or  an  arm  and  a  leg;  is  very  quick  to  learn  and  very  good  at  activi- 
ties  like  juggling,  pinball,  sawing  a  piece  of  wood  in  a  jigsaw,  and  oper¬ 
ating  a  lathe;  and,  has  no  difficulty  doing  even  very  complex  tasks.  ^ 

average  eye-limb  coordination  is  fairly  good  at 
activities  like  juggl ing,  pinball,  sawing  a  piece  of  wood  in  a  jigsaw,  and 
operating  a  lathe,  but  may  take  some  time  to  initially  learn  such  activi- 
ties;  for  very  complex  tasks,  she/he  occasionally  may  have  difficulty  co¬ 
ordinating  arm  and  leg  movements  precisely. 

An  individual  LOW  on  EYE-LIMB  COORDINATION  may  never  be  able  to  master 
tasks  requiring  coordinated  movements  of  two  arms,  two  legs,  or  an  arm  and 

extended  practice,  he/she  may  not  be  very  good  at  activi- 
inra  a  piece  of  wood  in  a  jigLw,  or  operat- 


Precision 


the  ability  to  make  steady,  sure  hand  and  arm  movements.  This 
ability  IS  important  for  tracking  tasks,  such  as  one  might  encounter  in 
video  games  (e.g.,  pursuing  an  object  that  is  flying  through  space  or  mov¬ 
ing  through  a  maze,  trying  to  keep  an  "X"  or  a  crosshair  centered  on  a 
moving  target).  The  objects  in  these  tasks  may  move  in  either  predictable 
or  random  patterns.  To  perform  well  on  these  tasks,  the  individual  is 
required  to  make  precise,  smooth  hand  and  arm  movements  to  stay  on  target 
Precision  also  is  important  for  performing  tasks  that  require  the  indiJi-' 

dual  to  maintain  the  hand  and  arm  in  a  single,  steady  position  for  a  Iona 
period  of  time.  ^ 


An  individual  HIGH  on  PRECISION  is  able  to  track  targets  very  ac- 
curately,  and  is  capable  of  holding  a  hand  or  arm  in  a  steady  position  for 
a  long  period  of  time  without  making  even  slight  tremors  or  movements. 

An  individual  AVERAGE  on  PRECISION  is  able  to  track  targets  fairly 
accurately,  but  may  make  occasional  large  tracking  errors;  and,  can  hold  a 
hand  or  arm  in  a  steady  position  for  a  short  period  of  time,  but  will  be¬ 
come  significantly  less  steady  after  a  while. 

fav-noJc  2”  PRECISION  has  a  great  deal  of  difficulty  tracking 

targets  and  makes  many  tracking  errors;  and,  is  not  able  to  hold  a  hand  or 
arm  in  a  steady  position  for  even  a  short  period  of  time. 
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Movement  Judgment 


This  is  the  ability  to  judge  the  relative  speed  and  direction  of  one 
or  more  moving  objects.  This  ability  is  used  to  determine  where  an  object 
will  be  at  a  given  time,  or  when  two  objects  will  meet  or  pass  each  other 
For  example,  movement  judgment  is  important  for  video  games  that  require 
shooting  or  bombing  a  moving  target,  because  the  player  must  judge  when  the 
bomb  and  target  will  meet. 


An  individual  HIGH  on  MOVEMENT  JUDGMENT  nearly  always  can  determine 
where  a  moving  object  will  be  at  a  given  time,  and  nearly  always  can  deter¬ 
mine  when  two  moving  objects  will  hit  or  pass  each  other,  even  if  the  ob- 
jGct(s)  ar©  moving  quickly,  at  sharp  anglGS,  or  along  a  complex  curve. 

An  individual  AVERAGE  on  MOVEMENT  JUDGMENT  usually  can  determine  where 
a  moving  object  will  be  at  a  given  time,  but  may  have  some  difficulty  de¬ 
termining  when  two  moving  objects  will  hit  or  pass  each  other  if  the  ob- 
ject{s)  are  moving  very  quickly,  at  sharp  angles,  or  along  complex  curves. 


An  individual  LOW  on  MOVEMENT  JUDGMENT  cannot  determine  where  a  movinq 
Object  will  be  at  a  given  time,  and  has  great  difficulty  determining  when 
two  moving  objects  will  hit  or  pass  each  other,  even  when  the  object(s)  are 
moving  slowly  in  relatively  straight  lines.  ^  ^ 
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This  is  the  ability  to  use  one’s  hands  or  fingers  to  manipulate  very 
small  to  moderately  large  objects  with  speed  and  precision. 

The  individual  HIGH  on  HAND  AND  FINGER  DEXTERITY  is  able  to  use  the 
hands  and  fingers  to  turn,  manipulate,  and  maintain  control  of  even  very 
smsll  objscts,  such  as  scrsws  and  dalicata  jawclry  or  housahold  ornamants. 

*U  individual  with  AVERAGE  HAND  AND  FINGER  DEXTERITY  is  able  to  use 

the  hands  and  fingers  to  turn,  manipulate,  and  maintain  control  of  most 

objects,  but  may  have  some  difficulty  manipulating  very  small  or  fine  ob- 
jects. 

The  individual  LOW  on  HAND  AND  FINGER  DEXTERITY  has  difficulty  using 

the  hands  and  fingers  to  manipulate  objects--and  may  be  described  by  others 
as  klutzy.  j  ^ 


Physical  Strength 


This  attribute  refers  to  the  ability  to  apply  pressure  to  and  move  an 
object.  For  example,  physical  strength  is  required  to  lift,  push,  pull, 
^ncl  carry  objGCts,  turn  a  valve,  and  perform  pull-ups  or  push-ups 


An  individual  HIGH  on  PHYSICAL  STRENGTH  is  able  to  push  and  pull  even 
very  heavy  objects;  has  no  difficulty  turning  even  tight,  large  valves;  can 
quickly  unload  heavy  bags  from  a  truck;  and,  can  pull  herself/himself  up  a 
rope  to  reach  a  ledge  or  platform  quickly. 


An  individual  with  AVERAGE  PHYSICAL  STRENGTH  is  able  to  push  and  pull 
moderately  heavy  objects;  may  have  some  difficulty  turning  large  or  tight 
valves;  can  unload  moderately  heavy  bags  from  a  truck  in  reasonable  time; 
and,  may  have  some  difficulty  pulling  herself/himself  up  a  rope  to  reach  a 
ledge  or  platform. 


./f’J.ndividual  LOW  on  PHYSICAL  STRENGTH  is  able  to  push  and  pull  only 
light  objects;  has  great  difficulty  turning  valves;  cannot  unload  heavy 
bags  from  a  truck;  and,  cannot  pull  herself/himself  up  a  rope  to  reach  a 
ledge  or  platform. 
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anJ''shovei'iJrd1rrS?"snor"*"® 

An  individual  HIGH  on  PHYSICAL  ENDURANCE  can  perform  physically  active 
work  for  long  periods  of  time  without  getting  tired;  can  perform^ 
repetitions  of  a  physical  task  without  stopping,  such  as  Linqina  an^ax- 
can  climb  even  long  flights  of  stairs  withSSt  ?esi"Sg:  andriJiriloJe 
staimna  than  his  or  hor  co-workors. 

An  individual  with  AVERAGE  PHYSICAL  ENDURANCE  can  perform  physically 
active  work  for  moderate  periods  of  time  without  getting  tired;  can  perform 

few^flinht?®nf^ir^  a  physical  task,  such  as  swinging  an  ax;  can  climb  a 
his  w  h“  cS-wHers."  “ 

individual  LOW  on  PHYSICAL  ENDURANCE  becomes  tired  after  short 
periods  of  physically  active  work;  must  rest  after  only  a  few  repetitions 
hrp^h^frlm  ^  task,  such  as  swinging  an  ax;  must  stop  to  catch  his  or  her 
her  CO  worker^^^  climbing  stairs;  and,  has  less  stamina  than  his  or 


This  attribute  refers  to  the  ability  to  keep  the  body  steady  while  at 
rest  or  in  motion,  and  to  perform  tasks  requiring  flexing  or  bending  For 
example,  balance  and  flexibility  are  required  for  walking  on  a  narrow  oath 

Sf  to  fit  through  or  i„?o  a  tight  area,  InS  stre?S?ng'^'’ 

reading  to  grasp  an  object. 

An  individual  HIGH  on  BALANCE  AND  FLEXIBILITY  can  keep  his/her  body 
steady  when  standing  on  a  small  or  slanted  surface,  such  as  a  ledge  or 
roof;  can  quickly  walk  or  crawl  across  a  narrow  plank  or  catwalk;  can  per¬ 
form  tasks  while  in  awkward  or  uncomfortable  positions  easily  and  quickly 
such  as  lying  in  a  tunnel  and  tightening  a  bolt  overhead;  and,  never  loses 
his/her  balance  while  bending,  twisting,  stretching,  or  reaching. 

with  AVERAGE  BALANCE  AND  FLEXIBILITY  has  only  minor 
hiyher  body  steady  when  standing  on  a  small  or  slanted 

generally  cross  a  narrow  plank  or 

•  Without  losing  balance;  may  have  some  difficulty  performing  tasks 

tiahtPninn'^a’^hoit°’^  Uncomfortable  positions,  such  as  lying  in  a  tunnel  and 

ihiil  and,  generally  does  not  lose  his/her  balance 

while  bending,  twisting,  stretching,  or  reaching. 

An  individual  LOW  on  BALANCE  AND  FLEXIBILITY  has  great  difficulty 

cnrh’a?  Steady  when  standing  on  a  small  or  slanted  surface, 

such  as  a  ledge  or  roof;  cannot  cross  a  narrow  plank  or  catwalk  without 

lScomfortIhlo®nn5-t-^’'®®^  difficulty  performing  tasks  while  in  awkward  or 
uncomfortable  positions,  such  as  lying  in  a  tunnel  and  tightening  a  bolt 


Involvement  in  Athletics 


This  attribute  refers  to  how  often  an  individual  participates  in 
physical  activities. 

An  Individual  HIGH  on  INVOLVEMENT  IN  ATHLETICS  participates  in  indi- 
Vidua  I  and  team  sports  and/or  exercises  vigorously  several  times  per  week. 

An  individual  with  AVERAGE  INVOLVEMENT  IN  ATHLETICS  participates  in 
individual  and  team  sports  and/or  exercises  once  or  twice  per  week. 

An  individual  LOW  on  INVOLVEMENT  IN  ATHLETICS  does  not  regularly  exer- 
cise  and/or  participate  in  individual  and  team  sports;  prefers  watching 
athletic  events  to  being  an  active  participant. 
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Work  Orientation 


n  4..  attribute  that  reflects  a  belief  that  hard  work  and 

sticking  to  it  pay  off.  It  is  characterized  by  a  belief  that  things  that 
happen  to  a  person  are  due  to  one’s  own  efforts  (i.e.,  personal  responsibi¬ 
lity)  rather  than  chance  events,  "fate,"  or  what  someone  else  does. 

An  individual  HIGH  on  WORK  ORIENTATION  always  tries  to  become  excel¬ 
lent  at  doing  his/her  job;  works  very  hard;  sets  extremely  high  work  stan¬ 
dards  and  almost  always  achieves  them;  always  tries  to  do  a  good  job  even 
when  assigned  unpleasant  tasks;  has  remarkably  high  concentration  and  real- 
ly  sticks-to-it  when  working  on  a  task;  believes  firmly  that  "you  get 
what  you  deserve"  and  that  people  control  what  happens  to  them;  and,  always 
takes  responsibility  for  her/his  work  and  life.  ^ 

An  individual  with  AVERAGE  WORK  ORIENTATION  usually  tries  to  become  a 
good  worker;  works  fairly  hard;  sets  moderate  work  standards  and  generally 
tries  to  do  a  good  job,  but  may  not  try  as  hard  when  assigned  unpleasant 
tasks;  concentrates  fairly  well  and  generally  "sticks  to"  a  task;  believes 

happens  to  them  but  makes  allowances  for 
outside  influence;  and,  usually  takes  responsibility  for  her/his  work  and 

An  individual  LOW  on  WORK  ORIENTATION  does  not  try  to  become  even  an 
acceptable  worker;  is  not  a  hard  worker;  sets  less  tha?i  acceptable  work 

^J!;®Iy„concentrates  or  "sticks  to"  a  task  until  completion;  does 
not  believe  that  you  get  what  you  deserve,"  rather  believes  that  what 

happens  to  people  is  beyond  their  personal  control;  and;  generally  does  not 
take  responsibility  for  her/his  work  and  life.  a^nerany  aoes  not 
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Sociabn  itv 


•  attribute  reflects  how  well  one  gets  along  in  social  situations. 
It  includes  being  outgoing,  talkative,  and  friendly,  and  being  comfortable 
in  group  situations. 

The  individual  HIGH  on  SOCIABILITY  is  outgoing,  talkative,  relates 
easily  to  others,  and  is  very  responsive  and  expressive  in  social  environ¬ 
ments;  has  many  friends  and  makes  new  friends  easily;  and,  is  comfortable 
in  group  situations. 

The  individual  AVERAGE  on  SOCIABILITY  is  somewhat  outgoing  and  talk- 

quiet  and  a  little  shy  in  large  groups;  may  find 
it  difficult  to  think  of  things  to  talk  about  around  new  friends  and 
acquaintances;  and,  neither  seeks  out  or  avoids  group  situations. 

The  individual  LOW  on  SOCIABILITY  is  quiet,  reserved,  and  shy; 
generally  avoids  getting  together  with  large  groups;  typically  has  only  a 
few  close  friends;  and,  typically  prefers  spending  time  alone. 
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Cooperation/Stabil  itv 

This  is  the  tendency  to  be  pleasant  in  dealing  with  others,  to  be 
tolerant,  tactful,  and  helpful,  and  to  maintain  control  over  one’s  emo¬ 
tions,  even  when  provoked. 

The  individual  HIGH  on  COOPERATION/STABILITY  is  very  pleasant  to  deal 
with  and  is  calm  even  during  stressful  situations;  is  tactful,  tolerant, 
helpful,  and  very  easy  to  get  along  with;  does  not  become  upset  or  hostile 
when  criticized  or  teased;  and,  maintains  an  even,  generally  happy  mood  al¬ 
most  all  of  the  time. 


The  individual  AVERAGE  on  COOPERATION/STABILITY  generally  is  pleasant 
to  doal  with,  but  may  bocom©  unplGasant  and  tGnse  during  V6ry  stressful 
situations;  is  tactful  and  easy  to  get  along  with  most  of  the  time,  but  may 
occasionally  become  upset  or  hostile  when  criticized  or  teased;  and 
usually  maintains  an  even  mood. 


The  individual  LOW  on  COOPERATION/STABILITY  is  generally  very  diffi- 
cult  to  deal  with,  especially  during  stressful  situations;  lacks  tact,  is 
intolerant  of  faults  in  others,  is  not  very  helpful  to  friends  or  acquain- 
tances,  and  does  not  get  along  well  with  others;  becomes  extremely  upset  or 
hostile  when  criticized;  has  frequent  changes  in  mood;  and,  often  becomes 
very  angry  and  upset  for  no  apparent  reason. 
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Energy 


This  attribute  refers  to  the  level  of  energy  and  enthusiasm  shown  by  a 
person. 


An  individual  HIGH  on  ENERGY  has  the  energy 
done;  is  nearly  always  enthusiastic,  optimistic, 
active  life-style. 


and  vitality  to  get  things 
and  cheerful;  and,  has  an 


An  individual  with  AVERAGE  ENERGY  has  the  energy  and  vitality  to  get 
most  of  her/his  responsibilities  completed,  but  may  not  have  the  energy  to 
work  extra  hours  or  meet  an  emergency;  occasionally  lacks  enthusiasm;  is 
sometimes  pessimistic  and  depressed;  and,  has  a  somewhat  active  life-style. 

An  individual  LOW  on  ENERGY  often  does  not  have  the  energy  to  complete 
assigned  tasks;  is  generally  pessimistic  and  depressed;  is  rarely  en¬ 
thusiastic;  and,  leads  an  inactive  life. 
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Conscientiousness 


This  is  the  attribute  that  reflects  respect  for  discipline,  order 
structure,  regulations,  and  authority.  It  results  in  planful,  dependable, 
well -organized  behavior,  stability,  and  resistance  to  change 


An  individual  HIGH  on  CONSCIENTIOUSNESS  accepts  authority  and  the  need 
for  and  value  of  discipline;  is  conventional  regarding  social  and  political 
views;  resists  social  change;  seeks  order  in  her/his  life;  has  a  high 

degree  of  self-control;  thinks  before  acting;  and,  is  always  honest  and 
I aw-abiding. 


An  individual  AVERAGE  on  CONSCIENTIOUSNESS  generally  accepts  authority 
and  the  value  of  discipline;  is  not  extreme  in  social  and  political  views; 
IS  willing  to  work  with  the  system  to  affect  planned  social  change;  typi- 
cally  plans  prior  to  taking  action  but  can  be  spontaneous;  has  moderate 
self-control;  and,  is  generally  honest  and  law-abiding. 

An  individual  LOW  on  CONSCIENTIOUSNESS  does  not  accept  authority  or 
the  need  for  and  value  of  discipline;  is  unconventional  in  his/her  social 
and  political  views;  enjoys  change  and  dislikes  order  in  her/his  life; 
lacks  self-control;  acts  before  thinking;  may  have  a  history  of  problems 
with  schools  and  law  enforcement  agencies;  and,  tends  to  be  dishonest. 
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This  is  the  tendency  to  seek  positions  of  leadership  and  influence 
over  others.  It  reflects  confidence  in  one’s  abilities  and  an  expectation 
that  one  can  succeed. 

The  individual  HIGH  on  DOMINANCE/CONFIDENCE  enjoys  being  a  leader  and 
seeks  leadership  positions;  can  be  highly  forceful  and  persuasive  when  the 
situation  requires;  has  succeeded  in  many  previous  undertakings  and  is  very 
confident  of  success  in  the  future.  ^ 

The  individual  with  AVERAGE  DOMINANCE/CONFIDENCE  may  enjoy  being  a 
leader,  but  generally  is  not  persuasive;  has  moderate  confidence  in  his/her 
aDinties;  and,  is  usually  confident  of  success  in  the  future. 

The  individual  LOW  on  DOMINANCE/CONFIDENCE  does  not  seek  leadership 
positions;  is  timid  about  offering  opinions,  advice,  or  suggestions;  often 
feels  incapable;  and,  has  many  doubts  about  succeeding  in  the  future. 
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nrartIJli  ®*rj?  ®  preference  for  physically  active  work  and 

practical,  concrete  activities.  Individuals  who  express  this  attribute 
enjoy  maintaining  and  repairing  machines  or  engines,  driving  cars  or 
trucks,  and  constructing  buildings  or  roads. 


Interest  in  Rugged  Activit.iP<; 


This  attribute  reflects  a 
outdoors,  and  using  a  weapon, 
hunting,  fishing,  camping,  and 


preference  for  rugged  activities,  working 
Individuals  who  express  this  attribute  enjoy 
hiking  in  the  wilderness. 
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Interest  in  Protective  Servirp^; 


This  attribute  reflects  a  preference  for  activities  that  involve  pro¬ 
tecting  people  and  property  from  harm.  Individuals  who  express  this  attri- 
safety”^°^  guarding  property,  fighting  fires,  enforcing  laws,  and  promoting 
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Interest  in  Technical  Ar.t.ivities 

This  attribute  reflects  a  preference  for  activities  that  involve  tech¬ 
nical  equipment  or  skills.  Individuals  who  express  this  attribute  eniov 
reading  and  drawing  blueprints  and  reading  aboSt  electronicf  ThlsfiSdi 
viduals  like  to  build,  repair,  and  operate  radios,  and  wSu^d  enjorope?- 
ating  movie  or  television  cameras  and  sound  equipment  ^ 
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Interest  in  Science 


This  attribute  reflects  a  preference  for  scholarly,  intellectual,  and 
scientific  activities.  Individuals  who  express  this  attribute  enjoy 
reading  scientific  or  scholarly  material  and  prefer  to  work  independently. 
These  individuals  enjoy  doing  research,  and  like  to  think  about  and  solve 
puzzling  problems,  such  as  diagnosing  a  disease  or  finding  an  error  in  a 
computer  program. 
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Interest  in  Leadershin 

This  attribute  reflects  a  preference  for  teaching,  helpinq  oersuad- 

''in’charai"^and^to^^®’^^‘ •  '^•^o  express  this  attribute  like  to  be 

n  charge  and  to  organize  a  group  and  mold  it  into  an  efficient  team 

These  individuals  enjoy  giving  speeches,  inspiring  others  to  excel  and 

TiSrs  viewpoint,  and  are  likely  to  be  g^od 
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Interest  in  Artistic  Activit.iP<; 


This  attribute  reflects  a  preference  for  unstructured,  expressive 
creative  activities.  Individuals  who  express  this  attribute  enjoy  readinq 
poetry  and  short  stories.  These  individuals  prefer  to  listen  to  classical 
music,  enjoy  ballet,  and  like  to  visit  art  museums. 
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Interest  in  Efficiency  and  Organization 


This  attribute  reflects  a  preference  for  well-ordered  and  systematic 
activities.  Individuals  who  express  this  attribute  enjoy  keeping  records, 
ordering  and  keeping  track  of  supplies,  and  working  with  tables  of  numbers. 
These  individuals  like  to  follow  a  list  of  steps  in  completing  a  task. 

They  enjoy  preparing  and  serving  food,  and  keeping  things  clean  and 
orderly. 
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PERFORMANCE  DEFINITION 


In  completing  your  ratings,  five  general  areas  of  performance  should 
be  kept  in  mind:  (1)  core  technical  proficiency,  (2)  general 
soldiering  proficiency,  (3)  effort  and  leadership,  (4)  personal 
discipline,  and  (5)  physical  fitness  and  military  bearing.  These 
performance  areas  are  defined  below. 

Core  Technical  Proficiency:  This  peformance  area  represents  the 
proficiency  with  which  the  soldier  performs  the  tasks  that  are 
"central"  to  the  MOS.  The  tasks  represent  the  core  of  the  job  and  are 
the  primary  definers  of  the  MOS.  This  performance  area  does  not 
include  the  individual’s  willingness  to  perform  the  task  or  the  degree 
to  which  the  individual  can  coordinate  efforts  with  others.  It  refers 
to  how  well  the  individual  can  execute  the  core  technical  tasks  the 
job  requires,  given  a  willingness  to  do  so. 

General  Soldiering  Proficiency:  In  addition  to  the  core  technical 
content  specific  to  an  MOS,  individuals  in  every  MOS  also  are 
responsible  for  being  able  to  perform  a  variety  of  general  soldiering 
tasks.  For  example,  determines  grid  coordinates  on  military  maps; 
puts  on,  wears,  and  removes  M17  series  protective  mask  with  hood; 
determines  a  magnetic  azimuth  using  a  compass;  collects  and  reports 
information;  and  recognizes  and  identifies  friendly  and  threat 
aircraft.  Performance  of  this  area  represents  overall  proficiency  on 
these  and  other  general  soldiering  tasks.  Again,  it  refers  to  how 
well  the  individual  can  execute  general  soldiering  tasks,  given  a 
willingness  to  do  so. 

Effort  and  Leadership:  This  performance  area  reflects  the  degree  to 
which  the  individual  exerts  effort  over  the  full  range  of  job  tasks, 
perseveres  under  adverse  or  dangerous  conditions,  and  demonstrates 
leadership  and  support  toward  peers.  That  is,  can  the  individual  be 
counted  on  to  carry  out  assigned  tasks,  even  under  adverse  conditions, 
to  exercise  good  judgment,  and  to  be  generally  dependable  and 
proficient?  While  appropriate  knowledges  and  skills  are  necessary  for 
successful  performance,  this  area  is  only  meant  to  reflect  the 
individual’s  willingness  to  do  the  job  required  and  to  be  cooperative 
and  supportive  with  other  soldiers. 

Personal  Discipline:  This  performance  area  reflects  the  degree  to 
which  the  individual  adheres  to  Army  regulations  and  traditions, 
exercises  personal  self-control,  demonstrates  integrity  in  day-to-day 
behavior,  and  does  not  create  disciplinary  problems.  People  who  rank 
high  on  this  area  show  a  commitment  to  high  standards  of  personal 
conduct. 

Physical  Fitness  and  Military  Bearing:  This  performance  area 
represents  the  degree  to  which  the  individual  maintains  an  appropriate 
military  appearance  and  bearing  and  stays  in  good  physical  condition. 
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Areas  of  Job  Performance  for  Soldiers 
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IIBIO  -  INFANTRYMAN 


Battlefield  Survival 

Performs  preventive  maintenance  and  assists  in  organizational  maintenance 
on  weapons  and  equipment. 

Protects  self,  weapons,  and  equipment  from  chemical  and  other  contaminants. 
Carries  and  prepares  ammunition  for  use  and  loads  weapons. 

Administers  first  aid. 

Applies  field  sanitation  methods. 

Insures  highest  state  of  physical  readiness  at  all  times. 

Combat  Techniques  and  Tactics 
Lays  field  mines. 

Performs  basic  communications  funtions  and  operates  platoon  communications 
equipment. 

Employs  cover,  concealment,  and  camouflage. 

Assists  in  construction  of  fortifications  and  barriers,  including  mine  fields 
Prepares  simple  demolitions. 

Assists  in  breaching  and  clearing  mine  fields  and  obstacles. 

Performs  indiredt  and  aerial  fire  support. 

Captures  prisoners  and  renders  verbal  reports  on  activities. 

Reacts  to  oral  commands  and  signals. 

Applies  principles  of  escape  and  evasion. 

Collects  and  reports  tactical  information  as  a  member  of  combat  or  recon¬ 
naissance  patrol. 

Performs  tactical  associated  duties  in  support  of  offensive,  defensive,  and 
specialized  missions. 

Weapons 

Uses  individual  infantry  weapons  and  associated  equipment. 

Carries,  emplaces,  sights,  and  fires  machine  gun. 

Throws  grenades  and  operates  grenade  launcher. 

Engages  in  hand-to-had  combat  and  employs  bayonet  and  silent  weapons. 
Carriers,  sights,  and  fires  light  antiarmor  weapons. 

Tactical  Vehicle/Supoort  Activities 

Operates  wheeled  and  tracked  truck  to  transport  personnel,  supplies,  and 
equipment. 

Performs  and  assists  in  unit  and  organizational  maintenance. 

Performs  duties  as  guard. 

Delivers  messages  and  performs  other  elementary  tasks  in  support  of  opera¬ 
tions  and  intelligence  functions. 

Performs  drill  and  ceremonies  and  other  post,  camp,  and  station  duties  as 
required. 

Maintenance 


Conducts  preventive  maintenance  checks  and  services  (PMCS)  on  organiza¬ 
tional  equipment. 

Driving 

Operates  track  vehicles  over  varied  terrain  in  varied  visibility  and  weather 
conditions,  using  techniques  of  movement  commensurate  with  the  threat. 
Reacts  to  audio  and  visual  signals. 

Assists  in  refueling  and  vehicle  operations. 
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63B10  -  LIGHT-WHEEL  VEHICLE  MECHANIC 


Repairs  and  Maintenance 

Performs  repairs  and  preventive  maintenance  checks  and  services  in  accor¬ 
dance  with  appropriate  maintenance  allocation  chart  (MAC). 

Tests  lead  acid  batteries  for  serviceability. 

Prepares  vehicles  and  equipment  for  operation  under  abnormal  conditions  by 
sealing,  waterproofing,  and  servicing  with  special  fuels,  coolants,  and 
lubricants. 

Employs  unit  maintenance  techniques  to  correct  malfunctions  of  mechanical, 
electrical,  and  hydraulic  brake  systems  or  components. 

Performs  preventive  maintenance  on  tools,  repair  parts,  and  equipment. 

Adjusts  operating  mechanisms. 

Tunes  engines  and  replaces  repair  parts  and  components  as  outlined  in  the 
appropriate  MAC. 

Tests  and  operates  vehicles  and  equipment  as  required  and  observes  for 
evidence  of  abnormal  operation. 

Operates,  or  assists  in  the  operation  of  wheel  vehicles  in  conjunction  with 
maintenance  and  recovery  operations. 

Employs  applicable  TMDE  in  conjunction  with  technical  publications  to 
troubleshoot  and  test  continuity  of  electrical  circuits. 

Interprets  schematic  diagrams. 

Determines  extent  of  corrective  action  and  repair  parts  required  to  correct 
malfunctions. 

Uses  hand  tools. 

Operates  power  tools  and  equipment. 

Obtains  replacement  parts  from  maintenance  parts  supply  section. 

Turns  in  unserviceable  parts  to  maintenance  parts  supply  section  while 
adhering  to  authorized  policies  and  required  procedures. 

Reads,  understands,  and  applies  information  contained  in  unit  maintenance 
technical  manuals  and  common  schematic  diagrams. 

Knows  and  applies  required  safety  procedures. 

Performs  or  assists  in  vehicle  recovery  and  evacuation  of  vehicles  that 
weigh  5  tons  or  less. 

Maneuvers  wrecker  to  desired  position. 

Uses  recovery  rigging  techniques  to  attach  and  secure  hoists,  cables,  and 
slings,  using  authorized  equipment  and  effects  recovery  operations. 

Performs  basic  oxy acetylene  cutting. 

Performs  operator  maintenance  on  light-wheel  wreckers. 

Knows  and  applies  recovery  operations  safety  procedures. 

Reads  topographic  maps. 

Uses  proper  tactical  radio  procedures. 
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71L10  -  ADMINISTRATIVE  SPECIALIST 


Typing 

Types  military  and  nonmilitary  correspondence,  messages,  orders,  recurring, 
and  special  reports,  requisitions,  forms,  regulations,  directives,  SOP, 
and  similar  materials. 

Types  material  in  draft  and  final  copy. 

Uses  written  notes,  drafts,  verbal  instructions,  or  other  sources  to  pre¬ 
pare  copy. 

Prepares  typewritten  copy  in  proper  format  to  comply  with  prescribed  corre- 
spondence  directives,  including  the  use  of  authorized  abbreviations. 

Employs  basic  principles  of  English  composition  and  grammar  in  preparing 
correspondence.  ^ 

Proofreads  typed  material  against  source  documents. 

Assembles  final  product  review,  signature,  authentication,  or  other  dispo¬ 
sition.  ^ 

Maintains  typewriters. 

Clerical 


Greets  office  visitors  and  directs  visitors  to  proper  destinations 
Announces  arrival  of  visitors. 

Answers  telephone  and  records  telephone  messages. 

Opens,  sorts,  routes,  and  delivers  incoming  correspondence  and  messages. 
Prepares  suspense  control  documents  and  maintains  suspense  files. 

Reproduces  copies  of  printed  material  using  copying  equipment. 

Signs  receipts  for  and  picks  up  registered  and  certified  mail. 

Prepares  file  labels  and  guides. 

Determines  proper  functional  file  and  file  numbers,  posts  to  documents,  and 
Tiles  documents  in  accordance  with  TAFFS. 

Destroys  or  disposes  of  files  in  accordance  with  disposition  instructions. 
Receives  publications  and  maintains  publications  library. 

Posts  changes  to  regulations  and  directives. 

Receives  and  stocks  blank  forms. 
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INSTRUCTIONS  FOR  VALIDITY  JUDGMENT 
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SECTION  I:  INTRODUCTION  TO  THE  JUDGMENT  TASK 


Today  you  will  be  asked  to  make  judgments  about  attributes,  job  perfor¬ 
mance,  and  the  relationship  between  the  two.  Let’s  start  witlr  some  key  def¬ 
initions. 

Attribute;  A  knowledge,  skill,  ability,  or  other  personal  characteristic 
such  as  interests,  values,  temperament,  etc.  Persons  can  be  tested  or 
evaluated  and  assigned  scores  on  attributes  (e.g.,  reading  comprehension, 
numerical  ability). 

Job  Performance:  The  way  that  a  person  carries  out  the  job  tasks  and  other 
requirements  of  the  job.  Persons  can  be  tested  or  evaluated  and  assigned 
scores  on  various  areas  of  job  performance  (e.g.,  physical  fitness  test 
scores,  AIT  grades,  instructors’  and  supervisors’  evaluations). 

Validity:  The  way  in  which  scores  on  an  attribute  are  related  to  scores  on 
job  performance.  For  example,  if  persons  scoring  low  on  an  attribute  also 

score  low  on  job  performance,  and  persons  scoring  high  on  that  same  attri¬ 

bute  also  score  high  on  job  performance,  then  that  attribute  has  high  val¬ 
idity.  But  if  scores  on  an  attribute  are  not  related  to  scores  on  job  per¬ 
formance,  then  there  is  little  or  no  validity  for  that  attribute.  For  ex¬ 
ample,  if  we  find  that  almost  all  persons  who  score  in  the  top  twenty  per¬ 
cent  on  a  writing  skills  test  are  also  consistently  the  best  newspaper  re¬ 

porters,  as  rated  by  their  supervisors,  and  low  scorers  on  the  test  are 
consistently  rated  the  worst  reporters,  then  writing  skill  is  valid  for 
newspaper  reporting. 


You  have  been  selected  to  make  these  judgments  because  you  are  familiar 
with  Army  job  performance.  In  completing  your  judgments,  think  about  the 
performance  of  soldiers  you  have  supervised  in  the  last  year  or  so.  You 
will  be  given  a  separate  handout  defining  job  performance  in  more  detail. 
Please  refer  to  it  when  making  your  judgments. 

You  will  be  asked  to  make  judgments  about  several  attributes.  These  attri¬ 
butes  are  defined  in  a  separate  handout,  and  descriptions  of  individuals 
scoring  high,  average,  and  low  on  each  attribute  are  provided.  The  attri¬ 
butes  include  cognitive  ability,  temperament  and  background,  interests,  and 
values. 


D39 


SECTION  II:  SPECIFIC  INSTRUCTIONS 


For  each  attribute,  judge  the  validity  of  the  attribute  for  performance  in 
each  of  the  five  job  areas  outlined  in  the  "Performance  Definition"  hand¬ 
out. 

As  discussed  above,  validity  indicates  the  relationship  between  scores  on  a 
test  of  the  attribute  and  scores  on  job  performance.  For  example,  if 
scores  on  the  attribute  are  not  related  to  job  performance  scores,  validity 
would  be  very  low.  This  is  the  case  when  individuals  who  all  have  high 
scores  on  the  test  have  job  performance  scores  ranging  from  very  low  to 
very  high.  In  this  low  validity  situation,  we  can  tell  nothing  about  how  a 
soldier  might  do  on  the  job  even  though  we  do  know  his  or  her  score  on  the 
attribute  test.  If  scores  on  the  attribute  are  closely  related  to  job  per¬ 
formance  scores,  validity  would  be  high.  This  is  the  case  when  scores  on 
the  test  very  closely  match  scores  on  job  performance  for  nearly  all  in¬ 
dividuals.  That  is,  nearly  all  those  getting  high  scores  on  the  attribute 
test  also  get  high  scores  on  job  performance  and  nearly  all  those  getting 
low  scores  on  the  test  get  low  scores  on  job  performance.  In  this  high 
validity  situation,  we  can  tell  a  whole  lot  about  how  well  a  soldier  will 
do  on  the  job  when  we  know  what  score  she/he  received  on  the  test  of  the 
attribute. 

Use  the  following  scale  to  make  the  validity  judgments  (this  scale  is  also 
shown  on  a  separate  handout): 


VALIDITY  JUDGMENT  SCALE 

Response  Level  of 

Option  Validity 


0 

1 

2 

3 

4 

5 

6 

7 

8 


no  validity 


low  validity 


medium  validity 


high  validity 


extremely  high  validity 
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Please  refer  to  the  sample  response  sheet  below  as  you  read  through  the 
following  examples  for  Verbal  Ability  and  Reasoning. 


Validity  for  Job  Area 


Verbal  Ability 


Reasoning 


Joe  supervises  95B  (Military  Police).  After  thoroughly  reading  the  perfor¬ 
mance  definition  and  the  definition  of  Verbal  Ability,  Joe  believes  that 
scores  on  a  Verbal  Ability  test  are  probably  highly  related  to  performance 
of  the  core  technical  tasks  of  a  95B.  So,  he  enters  a  6  in  the  column 
headed  Validity  for  Job  Area:  Core  Technical."  Joe  believes  that  scores 
on  a  Verbal  Ability  test  are  probably  moderately  related  to  performance 

soldiering  tasks,  so  he  enters  a  4  in  the  column  headed 
Validity  for  Job  Area:  General  Soldiering."  Joe  completes  the  rest  of 
the  job  area  validity  judgments  for  Verbal  Ability  in  a  similar  manner. 

Joe  th®n  proceeds  to  the  second  attribute.  Reasoning.  After  reading  the 
definition  of  Reasoning,  Joe  believes  that  scores  on  a  Reasoning  test  are 
probably  moderately  related  to  core  technical  task  performance  for  95Bs. 

t’®,,®'^^®’^^  column  headed  "Validity  for  Job  Area:  Core  Tech- 

nical.  Joe  thinks  scores  on  a  Reasoning  test  are  probably  somewhat  re¬ 
lated  to  general  soldiering  performance  scores,  so  he  enters  a  3  in  the 
column  headed  "Validity  for  Job  Area:  General  Soldiering."  Joe  completes 
the  rest  of  the  job  area  validity  judgments  in  a  similar  manner. 

Keep  the  "Performance  Definition"  and  the  rating  scale  handy  and  refer  to 
them  as  frequently  as  necessary  while  making  your  judgments. 


Jo  A  A  1 

^  S  2^  3  I 
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Graph  of  Low  Validity  Attribute 

Relationship  of  Attribute  to  Performance 


Attribute  Score 
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Graph  of  High  Validity  Attribute 

Relationship  of  Attribute  to  Performance 
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Perceptual  Speed  and  Accuracy 


RESPONSE  SHEET  FOR  VALIDITY  JUDGMENT;  COORDINATION  AND  PHYSICAL  ABILITIES 


Involvement  in  Athletics 


RESPONSE  SHEET  FOR  VALIDITY  JUDGMENT:  TEMPERAMENT/ATTITUDE 
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RESPONSE  SHEET  FOR  VALIDITY  JUDGMENT:  INTERESTS 


Mechanical  Comprehension 


RESPONSE  SHEET  FOR  VALIDITY  JUDGMENT:  COORDINATION  AND  PHYSICAL  ABILITIES 


Involvement  in  Athletics 


RESPONSE  SHEET  FOR  VALIDITY  JUDGMENT:  INTERESTS 


in  Efficiency  and  Organization 


INSTRUCTIONS  FOR  ATTRIBUTE  RANKING: 


VALIDITY 


In  this  task,  you  will  be  asked  to  rank  order  the  thirty-one  attributes 
according  to  their  validity  for  overall  performance  in  the  MOS  you  super¬ 
vise.  In  considering  overall  performance,  think  about  all  areas  of  job 
performance.  You  may  want  to  review  the  performance  definition  and  the 
attribute  definitions  before  beginning. 

Look  at  the  list  of  thirty-one  attributes.  Think  about  how  valid  each 
attribute  is  for  overall  performance  in  the  MOS  you  supervise.  Pick  the 
attribute  that  you  think  is  most  valid  and  place  a  "1"  in  the  space  beside 
the  attribute.  Next,  pick  the  attribute  that  you  think  is  least  valid  and 
place  a  "31"  beside  the  attribute.  Then,  pick  the  attribute  that  you  think 
is  second  most  valid  and  place  a  "2"  beside  the  attribute.  Next,  pick  the 
attribute  that  you  think  is  the  second  least  valid  and  place  a  "30"  to  the 
left  of  the  attribute.  Complete  the  rest  of  the  rank  orderings  in  this 
way,  alternating  back  and  forth  from  most  to  least  valid,  until  you  have 
ranked  all  thirty-one  attributes.  When  you  are  finished,  review  the  rank 
orders  to  be  certain  that  you  have  used  each  number  (1  through  31)  only 
once. 

Please  refer  to  the  diagrams  as  you  go  through  the  following  example. 


A.  Joe  supervises  95B,  Military  Police.  After  reviewing 
the  attribute  definitions  and  the  definition  of  job 
performance,  Joe  decides  that  Physical  Strength  is  the 
most  valid  attribute  for  overall  performance  as  a 
95B.  So,  Joe  puts  a  "1"  beside  Physical  Strength. 


Verbal  Ability 
Reasoning 
Number  Ability 
Spatial  Ability 
Closure 

Mental  Information 
Processing 

Perceptual  Speed 
and  Accuracy 

Memory 

Mechanical  Comprehension 
Eye- Li mb  Coordination 
Precision 


_  Movement  Judgment 

_  Hand  and  Finger  Dexterity 

Physical  Strength 

_  Physical  Endurance 

_  Balance  and  Flexibility 

_  Involvement  in  Athletics 

_  Work  Orientation 

_  Sociability 
_  Cooperation/Stability 
_  Energy 

_  Conscientiousness 
_  Dominance/Confidence 


Interest  in  Using 
Tools  and  Machines 

Interest  in  Rugged 
Activities 

Interest  in  Protec¬ 
tive  Services 

Interest  in  Technical 
Activities 

Interest  in  Science 

Interest  in  Leadership 

Interest  in  Artistic 
Activities 

Interest  in  Efficiency 
and  Organization 
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B.  Next,  Joe  decides  that  Interest  in  Artistic  Activities 
is  least  valid  for  overall  performance.  So,  he 
puts  a  "31"  beside  Interest  in  Artistic  Activities. 


Verbal  Ability 
Reasoning 
Number  Ability 
Spatial  Ability 
Closure 

Mental  Information 
Processing 

Perceptual  Speed 
and  Accuracy 

Memory 

Mechanical  Comprehension 
Eye-Limb  Coordination 
Precision 


_  Movement  Judgment 

_  Hand  and  Finger  Dexterity 

Physical  Strength 

_  Physical  Endurance 

_  Balance  and  Flexibility 

_  Involvement  in  Athletics 

_  Work  Orientation 

_  Sociability 

_  Cooperation/Stability 

_  Energy 

_  Conscientiousness 
_  Oominance/Confidence 


_  Interest  in  Using 

Tools  and  Machines 

_  Interest  in  Rugged 

Activities 

_  Interest  in  Protec¬ 
tive  Services 

_  Interest  in  Technical 

Activities 

_  Interest  in  Science 

_  Interest  In  Leadership 

Interest  in  Artistic 
Activities 

_  Interest  in  Efficiency 

and  Organization 


C.  Joe  then  decides  that  Verbal  Ability  is  the  second  most 
valid  attribute  for  overall  performance.  He  puts  a 
"2"  beside  Verbal  Ability. 


2^  Verbal  Ability 

_  Reasoning 

_  Number  Ability 

_  Spatial  Ability 

_  Closure 

_  Mental  Information 

Processing 

_  Perceptual  Speed 

and  Accuracy 

_  Memory 

_  Mechanical  Comprehension 

_  Eye-Limb  Coordination 

Precision 


_  Movement  Judgment 

_  Hand  and  Finger  Dexterity 

f  Physical  Strength 

_  Physical  Endurance 

_  Balance  and  Flexibility 

_ :  Involvement  in  Athletics 

_  Work  Orientation 

_  Sociability 

_  Cooperation/Stability 

_  Energy 

Conscientiousness 
_  Dominance/Confidence 


_  Interest  in  Using 

Tools  and  Machines 

_  Interest  in  Rugged 

Activities 

_  Interest  in  Protec¬ 
tive  Services 

_  Interest  in  Technical 

Activities 

_  Interest  in  Science 

_  Interest  in  Leadership 

Interest  in  Artistic 
Activities 

_  Interest  in  Efficiency 

and  Organization 
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D.  Next,  Joe  decides  that  Closure  is  the  second  least 
valid  attribute,  so  he  puts  a  "30"  beside  Closure. 


2,  Verbal  Ability 

_  Reasoning 

_  Number  Ability 

_  Spatial  Ability 

30  Closure 

_  Mental  Information 

Processing 

_  Perceptual  Speed 

and  Accuracy 

_  Memory 

_  Mechanical  Comprehension 

_  Eye-Limb  Coordination 

Precision 


_  Movement  Judgment 

_  Hand  and  Finger  Dexterity 

_J_  Physical  Strength 

_  Physical  Endurance 

_  Balance  and  Flexibility 

_  Involvement  In  Athletics 

_  Work  Orientation 

_  Sociability 

_  Cooperatlon/Stablllty 

_  Energy 

_  Conscientiousness 

Dominance/Confidence 


_  Interest  In  Using 

Tools  and  Machines 

_  Interest  in  Rugged 

Activities 

_  Interest  In  Protec¬ 
tive  Services 

_  Interest  in  Technical 

Activities 

_  Interest  in  Science 

_  Interest  In  Leadership 

^  Interest  in  Artistic 
Activities 

_  Interest  in  Efficiency 

and  Organization 


Joe  continues  to  rank  order  the  attributes  in  this  way, 
alternating  back  and  forth  between  most  and  least 
valid,  until  all  thirty-one  attributes  have  been  rank- 
ordered.  As  a  final  check,  Joe  makes  certain  that  he 
used  each  number  from  1  through  31  only  once. 
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ATTRIBUTE  RANKING  WORKSHEET 


Verbal  Ability 
Reasoning 
Number  Ability 
Spatial  Ability 
Closure 

Mental  Information 
Processing 

Perceptual  Speed 
and  Accuracy 

Memory 

Mechanical  Comprehension 
Eye-Limb  Coordination 
Precision 
Movement  Judgment 
Hand  and  Finger  Dexterity 
Physical  Strength 
Physical  Endurance 
Balance  and  Flexibility 
Involvement  in  Athletics 


Work  Orientation 

Sociability 

Cooperati on/Stabi 1 i ty 

Energy 

Conscientiousness 

Dominance/Confidence 

Interest  in  Using 
Tools  and  Machines 

Interest  in  Rugged 
Activities 

Interest  in  Protec¬ 
tive  Services 

Interest  in  Technical 
Activities 

Interest  in  Science 

Interest  in  Leadership 

Interest  in  Artistic 
Activities 

Interest  in  Efficiency 
and  Organization 


Please  check  to  see  that  you  used  each  number  from  1  through  31  only  once. 


INSTRUCTIONS  FOR  IMPORTANCE  JUDGMENT 
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SECTION  I:  INTRODUCTION  TO  THE  JUDGMENT  TASK 


Today  you  will  be  asked  to  make  judgments  about  attributes,  job  perfor¬ 
mance,  and  the  relationship  between  the  two.  Let’s  start  with  some  key 
definitions. 

Attribute:  A  knowledge,  skill,  ability,  or  other  personal  characteristic 
such  as  interests,  values,  temperament,  etc.  Persons  can  be  tested  or 
evaluated  and  assigned  scores  on  attributes  (e.g.,  reading  comprehension, 
numerical  ability). 

Job  Performance:  The  way  that  a  person  carries  out  the  job  tasks  and  other 
requirements  of  the  job.  Persons  can  be  tested  or  evaluated  and  assigned 
scores  on  various  areas  of  job  performance  (e.g.,  physical  fitness  test 
scores,  AIT  grades,  instructors’  and  supervisors’  evaluations). 

Importance:  The  degree  to  which  use  of  an  attribute  is  directly  involved 
in  job  performance.  For  example,  auto  mechanics  must  know  and  make  use  of 
information  about  how  internal  combustion  engines  work. 

You  have  been  selected  to  make  these  judgments  because  you  are  familiar 
with  Army  job  performance.  In  completing  your  judgments,  think  about  the 
performance  of  soldiers  you  have  supervised  in  the  last  year  or  so.  You 
will  be  given  a  separate  handout  defining  job  performance  in  more  detail. 
Please  refer  to  it  when  making  your  judgments. 

You  will  be  asked  to  make  judgments  about  several  attributes.  These  attri¬ 
butes  are  defined  in  a  separate  handout,  and  descriptions  of  individuals 
scoring  high,  average,  and  low  on  each  attribute  are  provided.  The  attri¬ 
butes  include  cognitive  ability,  temperament  and  background,  interests,  and 
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SECTION  II:  SPECIFIC  INSTRUCTIONS 


For  each  attribute,  rate  the  importance  of  the  attribute  for  performance  in 
each  of  the  five  job  areas  outlined  in  the  "Performance  Definition"  hand¬ 
out.  Use  the  following  scale  for  the  importance  judgments  (this  scale  is 
also  shown  on  a  separate  handout): 

IMPORTANCE  JUDGMENT  SCALE 

Response  Level  of 

Option  Importance 

0  no  importance 

1 

2  low  importance 

3 

A  moderate  importance 

5 

6  high  importance 

7 

8  extremely  high  importance 
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Please  refer  to  the  sample  response  sheet  below  as  you  read  through 
examples  for  Verbal  Ability  and  Reasoning. 


Attribute 


Importance  for  Job  Area 

Sr  -O'  "v  X, 

^ 

^  •o*'  .eP 


Verbal  Ability 
Reasoning 


^  ±  ^  A  ± 
±  2.  A  3  I 


Joe  supervises  95B  (Military  Police).  After  thoroughly  reading  the  perfor¬ 
mance  definition  and  the  definition  of  Verbal  Ability,  Joe  concludes  that 
Verbal  Ability  is  of  high  importance  for  success  on  the  core  technical 
tasks  a  95B  must  perform.  So,  Joe  enters  response  option  6,  for  "High  Im¬ 
portance,"  under  the  column  headed  "Importance  to  Job  Area:  Core  Tech¬ 
nical."  Joe  thinks  Verbal  Ability  is  moderately  important  for  general  sol¬ 
diering  tasks,  so  he  enters  option  4,  signifying  "Moderate  Importance," 
under  the  column  headed  "Importance  to  Job  Area:  General  Soldiering."  Joe 
thinks  that  Verbal  Ability  is  only  slightly  important  to  effort  and  leader¬ 
ship,  so  he  enters  a  2  in  the  column  headed  "Importance  to  Job  Area: 

Effort  &  Leadership."  Joe  completes  the  rest  of  the  judgments  for  the  five 
job  areas  in  a  similar  manner. 

Joe  then  proceeds  to  the  second  attribute.  Reasoning.  After  reading  the 
dafimtion  of  Reasoning,  Joe  decides  that  Reasoning  is  of  moderate  impor- 
tance  for  performance  of  the  95B  core  technical  tasks.  So,  he  enters  a  4 
in  the  column  headed  "Importance  to  Job  Area:  Core  Technical."  Joe  thinks 
Reasoning  is  somewhat  important  for  general  soldiering  tasks,  so  he  enters 
a  3  in  the  column  headed  "Importance  to  Job  Area:  General  Soldiering  " 

Joe  believes  Reasoning  is  of  low  importance  for  effort  and  leadership!  so 
he  enters  a  2  in  the  column  headed  "Importance  to  Job  Area:  Effort  & 
Leadership."  Joe  completes  the  rest  of  the  importance  judgments  for  the 
five  job  areas  in  a  similar  manner. 

Keep  the  "Performance  Definition"  and  rating  scale  handy  and  refer  to  them 
as  frequently  as  necessary  while  making  your  judgments. 
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Mechanical  Comprehension 


RESPONSE  SHEET  FOR  IMPORTANCE  JUDGMENT:  COORDINATION  AND  PHYSICAL  ABILITIES 
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Involvement  in  Athletics 


RESPONSE  SHEET  FOR  IMPORTANCE  JUDGMENT;  TEMPERAMENT/ATTITUDE 
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RESPONSE  SHEET  FOR  IMPORTANCE  JUDGMENT:  INTERESTS 
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Interest  in  Efficiency  and  Organization 


Mechanical  Comprehension 


RESPONSE  SHEET  FOR  IMPORTANCE  JUDGMENT:  COORDINATION  AND  PHYSICAL  ABILITIES 


Involvement  in  Athletics 


RESPONSE  SHEET  FOR  IMPORTANCE  JUDGMENT:  TEMPERAMENT/ATTITUDE 
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RESPONSE  SHEET  FOR  IMPORTANCE  JUDGMENT;  INTERESTS 


Interest  in  Efficiency  and  Organization 


INSTRUCTIONS  FOR  ATTRIBUTE  RANKING: 
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IMPORTANCE 


In  this  task,  you  will  be  asked  to  rank  order  the  thirty-one  attributes 
according  to  their  importance  for  overall  performance  in  the  MOS  you 
supervise.  In  considering  overall  performance,  think  about  all  areas  of 
job  performance.  You  may  want  to  review  the  performance  definition  and  the 
attribute  definitions  before  beginning. 

Look  at  the  list  of  thirty-one  attributes.  Think  about  how  important  each 
attribute  is  for  overall  performance  in  the  MOS  you  supervise.  Pick  the 
attribute  that  you  think  is  most  important  and  place  a  "1"  in  the  space 
beside  the  attribute.  Next,  pick  the  attribute  that  you  think  is  least 
important  and  place  a  "31"  beside  the  attribute.  Then,  pick  the  attribute 
that  you  think  is  second  most  important  and  place  a  "2"  beside  the  attri¬ 
bute.  Next,  pick  the  attribute  that  you  think  is  the  second  least  imnor- 
tant  and  place  a  "30"  to  the  left  of  the  attribute.  Complete  the  rest  of 
the  rank  orderings  in  this  way,  alternating  back  and  forth  from  most  to 
least  important,  until  you  have  ranked  all  thirty-one  attributes.  When  you 
are  finished,  review  the  rank  orders  to  be  certain  that  you  have  used  each 
number  (1  through  31)  only  once. 

Please  refer  to  the  diagrams  as  you  go  through  the  following  example. 


A.  Joe  supervises  95B,  Military  Police.  After  reviewing 
the. attri bute  definitions  and  the  definition  of  job 
performance,  Joe  decides  that  Physical  Strength  is  the 
^^ost  important  attribute  for  overall  performance  as  a 
95B.  So,  Joe  puts  a  "1"  beside  Physical  Strength. 


_  Verbal  Ability 

_  Reasoning 

_  Number  Ability 

_  Spatial  Ability 

_  Closure 

_  Mental  Information 

Processing 

_  Perceptual  Speed 

and  Accuracy 

_  Memory 

_  Mechanical  Comprehension 

_  Eye-Limb  Coordination 

Precision 


_  Movement  Judgment 

_  Hand  and  Finger  Dexterity 

t  Physical  Strength 

_  Physical  Endurance 

_  Balance  and  Flexibility 

_  Involvement  In  Athletics 

_  Work  Orientation 

_  Sociability 

_  Cooperatlon/Stablllty 

_  Energy 

Conscientiousness 
_  Dominance/Confidence 


_  Interest  In  Using 

Tools  and  Machines 

_  Interest  in  Rugged 

Activities 

_  Interest  In  Protec¬ 
tive  Services 

_  Interest  in  Technical 

Activities 

_  Interest  in  Science 

_  Interest  in  Leadership 

_  Interest  in  Artistic 

Activities 

_  Interest  in  Efficiency 

and  Organization 
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B.  Next,  Joe  decides  that  Interest  in  Artistic  Activities 
is  least  important  for  overall  performance.  So,  he 
puts  a  "31"  beside  Interest  in  Artistic  Activities. 


_  Verbal  Ability 

_  Reasoning 

_  Number  Ability 

_  Spatial  Ability 

_  Closure 

_  Mental  Information 

Processing 

_  Perceptual  Speed 

and  Accuracy 

_  Memory 

_  Mechanical  Comprehension 

_  Eye-Limb  Coordination 

Precision 


_  Movement  Judgment 

_  Hand  and  Finger  Dexterity 

/  Physical  Strength 

_  Physical  Endurance 

_  Balance  and  Flexibility 

_  Involvement  1n  Athletics 

_  Work  Orientation 

_  Sociability 

_  Cooperation/Stability 

_  Energy 

_  Conscientiousness 

__  Dominance/Confidence 


C,  Joe  then  decides  that  Verbal  Ability  is  the 
important  attribute  for  overall  performance. 
"2"  beside  Verbal  Ability. 


2^  Verbal  Ability 

_  Movement  Judgment 

_  Reasoning 

_  Hand  and  Finger  Dexterity 

_  Number  Ability 

/  Physical  Strength 

_ Spatial  Ability 

_  Physical  Endurance 

_  Closure 

_  Balance  and  Flexibility 

_  Mental  Information 

_  Involvement  In  Athletics 

Processing 

_  Work  Orientation 

_  Perceptual  Speed 

and  Accuracy 

_  Sociability 

_  Memory 

_  Cooperati on/StabI 1 1 ty 

_  Mechanical  Comprehension 

_  Energy 

_  Eye-Limb  Coordination 

_  Conscientiousness 

_  Precision 

_  Doralnance/Confidence 

_  Interest  In  Using 

Tools  and  Machines 

_  Interest  In  Rugged 

Activities 

_  Interest  In  Protec¬ 
tive  Services 

_  Interest  In  Technical 

Activities 

_  Interest  In  Science 

_  Interest  In  Leadership 

a  Interest  in  Artistic 
Activities 

_  Interest  in  Efficiency 

and  Organization 


second  most 
He  puts  a 


_  Interest  in  Using 

Tools  and  Machines 

_  Interest  In  Rugged 

Activities 

_  Interest  in  Protec¬ 
tive  Services 

_  Interest  In  Technical 

Activities 

_  Interest  in  Science 

_  Interest  In  Leadership 

Interest  In  Artistic 
Activities 

_  Interest  In  Efficiency 

and  Organization 
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D.  Next,  Joe  decides  that  Closure  is  the  second  least 
important  attribute,  so  he  puts  a  "30"  beside  Closure. 


0~  Verbal  Ability 

_  Reasoning 

_  Number  Ability 

_  Spatial  Ability 

Closure 

_  Mental  Information 

Processing 

_  Perceptual  Speed 

and  Accuracy 

_  Memory 

_  Mechanical  Comprehension 

_  Eye- LI mb  Coordination 

_  Precision 


_  Movement  Judgment 

_  Hand  and  Finger  Dexterity 

Physical  Strength 

_  Physical  Endurance 

_  Balance  and  Flexibility 

_  Involvement  in  Athletics 

_  Work  Orientation 

_  Sociability 

_  Cooperation/Stability 

_  Energy 

_  Conscientiousness 

_  Dorainance/Confidence 


_  Interest  in  Using 

Tools  and  Machines 

_  Interest  in  Rugged 

Activities 

_  Interest  in  Protec¬ 
tive  Services 

_  Interest  in  Technical 

Activities 

_  Interest  in  Science 

_  Interest  in  Leadership 

Interest  in  Artistic 
Activities 

—  Interest  in  Efficiency 
and  Organization 


Joe  continues  to  rank  order  the  attributes  in  this  way, 
alternating  back  and  forth  between  most  and  least  im¬ 
portant,  until  all  thirty-one  attributes  have  been 
rank-ordered.  As  a  final  check,  Joe  makes  certain  that 
he  used  each  number  from  1  through  31  only  once. 
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ATTRIBUTE  RANKING  WORKSHEET 


Verbal  Ability 
Reasoning 
Number  Ability 
Spatial  Ability 
Closure 

Mental  Information 
Processing 

Perceptual  Speed 
and  Accuracy 

Memory 

Mechanical  Comprehension 
Eye-Limb  Coordination 
Precision 
Movement  Judgment 
Hand  and  Finger  Dexterity 
Physical  Strength 
Physical  Endurance 
Balance  and  Flexibility 
Involvement  in  Athletics 


Work  Orientation 

Sociability 

Cooperation/Stability 

Energy 

Conscientiousness 

Dominance/Confidence 

Interest  in  Using 
Tools  and  Machines 

Interest  in  Rugged 
Activities 

Interest  in  Protec¬ 
tive  Services 

Interest  in  Technical 
Activities 

Interest  in  Science 

Interest  in  Leadership 

Interest  in  Artistic 
Activities 

Interest  in  Efficiency 
and  Organization 


Please  check  to  see  that  you  used  each  number  from  1  through  31  only  once 
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Rated  MOS 


Your  Rank 


Today’s  Date 


Method  _  Name _ _ _ 

ATTRIBUTE  MODEL  EVALUATION  SHEET 
Performance  Definition 

Please  recall  (review,  if  necessary)  the  materials  used  to  define  perfor¬ 
mance  in  the  MOS.  Then,  answer  the  following  questions. 

1.  How  would  you  describe  the  amount  of  detail  and  information  provided  in 
the  performance  definition? 

1  2  3  4  5  6  7 

Far  Too  Little  The  Right  Far  Too  Much 

Amount 

2.  If  you  think  the  performance  definition  has  too  little  detail  and  in¬ 
formation,  what  would  you  add? 


3.  If  you  think  the  performance  definition  has  too  much  detail  and  infor¬ 
mation,  what  would  you  delete? 


Attribute  Ranking 


1.  How  clear  were  the  instructions  for  the  ranking? 


12  3  4 


5  6 


7 


Not  At 
All  Clear 


Neither  Clear 
Nor  Unclear 


Very 
Cl  ear 


2.  Was  it  easy  or  difficult  to  rank  order  the  attributes? 


1  2  3  4  5  6 


7 


Very 

Difficult 


Neither  Easy 
Nor  Difficult 


Very 

Easy 


3.  How  confident  do  you  feel  about  the  accuracy  of  your  rankings? 


1 

2 

3 

4  5 

6 

7 

Not  At  All 
Confident 

Somewhat 

Confident 

Very 

Confident 
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Attributes 


1.  Think  about  the  list  of  thirty-one  attributes  (below).  Place  a 

checkmark  in  front  of  any  attributes  that  were  difficult  to  judge  or 
rank.  Please  check  at  least  two  attributes. 


_  Verbal  Ability 

_  Reasoning 

_  Number  Ability 

_  Spatial  Ability 

_  Closure 

_  Mental  Information 

Processing 

_  Perceptual  Speed 

and  Accuracy 

_  Memory 

_  Mechanical  Comprehension 

_  Eye-Limb  Coordination 

_  Precision 

_  Movement  Judgment 

_  Hand  and  Finger  Dexterity 

_  Physical  Strength 

_  Physical  Endurance 

_  Balance  and  Flexibility 

_  Involvement  in  Athletics 


_  Work  Orientation 

_  Sociability 

_  Cooperation/Stability 

_  Energy 

_  Conscientiousness 

_  Dominance/Confidence 

_  Interest  in  Using 

Tools  and  Machines 

_  Interest  in  Rugged 

Activities 

_  Interest  in  Protec¬ 
tive  Services 

_  Interest  in  Technical 

Activities 

_  Interest  in  Science 

_  Interest  in  Leadership 

_  Interest  in  Artistic 

Activities 

_  Interest  in  Efficiency 

and  Organization 


2.  This  list  of  attributes  was  designed  to  cover  all  the  attributes  which 
are  relevant  to  Army  MOS. 

Are  there  any  attributes  you  would  add? 


Are  there  any  attributes  you  would  delete? 
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General  Evaluation 


Think  about  the  two  tasks  you  completed,  attribute  judgments  and  ranking. 
Then  answer  the  following  questions. 


1.  Which  was  easier? 

_  Judgment 

_  Ranking 

2.  Which  provides  more  information  about  the  MOS? 

_  Judgment 

_  Ranking 

3.  If  the  Army  was  going  to  use  one  of  these  methods  to  guide  selection 
and  placement  of  soldiers  in  MOS,  which  would  you  prefer? 

_  Judgment 

_  Ranking 
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APPENDIX  E 

DETAILED  RESULTS  OF  TASK  MATCHING  EXERCISE 


APPENDIX  E:  Napping  of  Project  A  Tasks  onto  Important  Task  Categories 
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APPENDIX  E:  Mapping  of  Project  A  Tasks  onto  Important  Task  Categories 
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APPENDIX  E:  Napping  of  Project  A  Tasks  onto  Important  Job  Activities 
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22.  Judge  size  and  distance 

23.  Judge  location 

24.  Judge  paths  of  moving  objects 
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APPENDIX  E:  Mapping  of  Project  A  Tasks  onto  Important  Job  Activities 
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APPENDIX  F 


SUMMARY  OF  EVALUATION  RATINGS  BY 
METHOD,  RATER  TYPE,  AND  MOS 


Summary  of  PERFDEFl  by  Method,  Rater  Type, 


and  MOS 


Variable 

Value  Label 

Mean 

Std  Dev 

Cases 

For  Total 

Sampl e 

4.55 

.96 

127 

METHOD 

Importance 

4.58 

.95 

53 

TYPE 

Civilian 

5.00 

0.0 

5 

MOS 

IIB 

5.00 

0.0 

1 

MOS 

63B 

5.00 

0.0 

3 

MOS 

71L 

5.00 

0.0 

1 

TYPE 

NCO 

4.48 

.95 

35 

MOS 

IIB 

4.65 

.93 

20 

MOS 

63B 

4.33 

.58 

3 

MOS 

71L 

4.25 

1.06 

12 

TYPE 

Officer 

4.69 

1.11 

13 

MOS 

IIB 

4.00 

0.0 

1 

MOS 

63B 

5.00 

1.41 

5 

MOS 

71L 

4.57 

.98 

7 

METHOD 

Validity 

4.53 

.98 

74 

TYPE 

Civilian 

4.50 

.55 

6 

MOS 

IIB 

4.50 

.55 

6 

TYPE 

NCO 

4.60 

1.22 

40 

MOS 

IIB 

4.31 

1.35 

16 

MOS 

63B 

4.65 

1.11 

17 

MOS 

71L 

5.14 

1.07 

7 

TYPE 

Off i cer 

4.43 

.63 

28 

MOS 

IIB 

4.57 

.76 

14 

MOS 

63B 

4.25 

.50 

4 

MOS 

71L 

4.30 

.48 

10 

F-1 


Summary  of  RANKl  by  Method, 

Rater  Type,  and 

MOS 

Variable 

Value  Label 

Mean 

Std  Dev 

Cases 

For  Total 

Sample 

5.61 

1.32 

127 

METHOD 

Importance 

6.17 

.78 

53 

TYPE 

Civilian 

5.40 

.55 

5 

MOS 

IIB 

5.00 

0.0 

1 

MOS 

63  B 

5.67 

.58 

3 

MOS 

71L 

5.00 

0.0 

1 

TYPE 

NCO 

6.23 

.81 

35 

MOS 

IIB 

6.25 

.79 

20 

MOS 

63B 

6.67 

.58 

3 

MOS 

71L 

6.08 

.90 

12 

TYPE 

Officer 

6.31 

.63 

13 

MOS 

IIB 

6.00 

0.0 

1 

MOS 

63B 

6.80 

.45 

5 

MOS 

71L 

6.00 

.58 

7 

METHOD 

Validity 

5.22 

1.47 

74 

TYPE 

Civilian 

4.33 

1.86 

6 

MOS 

IIB 

4.33 

1.86 

6 

TYPE 

NCO 

4.92 

1.58 

40 

MOS 

IIB 

4.31 

1.66 

16 

MOS 

63B 

5.41 

1.37 

17 

MOS 

71L 

5.14 

1.57 

7 

TYPE 

Officer 

5.82 

.98 

28 

MOS 

IIB 

5.57 

1.16 

14 

MOS 

63B 

6.00 

.82 

4 

MOS 

71L 

6.10 

.74 

10 

F-2 


Summary  of  RANK2  by  Method,  Rater  Type,  and  MOS 


Variable 

Value  Label 

Mean 

Std  Dev 

Cases 

For  Total 

Sample 

4.27 

1.59 

127 

METHOD 

Importance 

4.51 

1.67 

53 

TYPE 

Civilian 

4.40 

1.34 

5 

MOS 

IIB 

6.00 

0.0 

1 

MOS 

63B 

4.33 

1.15 

3 

MOS 

71L 

3.00 

0.0 

1 

TYPE 

NCO 

4.63 

1.77 

35 

MOS 

IIB 

4.75 

1.74 

20 

MOS 

63B 

3.00 

2.00 

3 

MOS 

71L 

4.83 

1.70 

12 

TYPE 

Officer 

4.23 

1.59 

13 

MOS 

IIB 

5.00 

0.0 

1 

MOS 

63  B 

4.60 

1.82 

5 

MOS 

71L 

3.86 

1.57 

7 

METHOD 

Validity 

4.09 

1.52 

74 

TYPE 

Civilian 

3.33 

1.21 

6 

MOS 

IIB 

3.33 

1.21 

6 

TYPE 

NCO 

4.28 

1.62 

40 

MOS 

IIB 

4.25 

1.53 

16 

MOS 

63B 

4.12 

1.69 

17 

MOS 

71L 

4.71 

1.80 

7 

TYPE 

Officer 

4.00 

1.41 

28 

MOS 

IIB 

4.00 

1.47 

14 

MOS 

63B 

3.00 

1.63 

4 

MOS 

71L 

4.40 

1.17 

10 

F-3 


Summary  of  RANKS  by  Method,  Rater  Type,  and  MOS 


Variable 

Value  Label 

Mean 

Std  Dev 

Cases 

For  Total 

Sample 

4.92 

1.31 

127 

METHOD 

Importance 

5.24 

1.04 

53 

TYPE 

Civilian 

5.00 

.71 

5 

MOS 

IIB 

6.00 

0.0 

1 

MOS 

63B 

4.67 

.58 

3 

MOS 

71L 

5.00 

0.0 

1 

TYPE 

NCO 

5.23 

1.19 

35 

MOS 

IIB 

5.15 

1.04 

20 

MOS 

63B 

5.00 

1.73 

3 

MOS 

71L 

5.42 

1.38 

12 

TYPE 

Officer 

5.38 

.65 

13 

MOS 

IIB 

6.00 

0.0 

1 

MOS 

63B 

5.60 

.55 

5 

MOS 

71L 

5.14 

.69 

7 

METHOD 

Val idity 

4.69 

1.44 

74 

TYPE 

Civilian 

4.83 

1.17 

6 

MOS 

IIB 

4.83 

1.17 

6 

TYPE 

NCO 

4.60 

1.55 

40 

MOS 

IIB 

4.44 

1.50 

16 

MOS 

63B 

4.53 

1.50 

17 

MOS 

71L 

5.14 

1.86 

7 

TYPE 

Officer 

4.78 

1.37 

28 

MOS 

IIB 

4.78 

1.37 

14 

MOS 

63B 

4.00 

2.16 

4 

MOS 

71L 

5.10 

.99 

10 
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XopDOving  tlie  SeLection  of  MDS  UH  TOW  Qmneacs 

Can  ^  use  a  selection  test  to  identify  soldiers  ^  will  be  good  IIH  TCW 
Gunners? 

InteseardLjOcus  NO.  1,  ^  r^rted  that  conputerized  and  paper-and-  # 
pencil  tests  developed  by  the  Acny's  Project  A  predict  M-1  gunnery  in  the 
Unit  Conduct  of  Fire  Trainer  (TOOFT).  m  this  issue,  we  present  results  of  a 
project,  conducted  for  the  OG  TOADDC,  which  shows  that  Project  A  tests 
predict  neasures  of  TCW  gunnery  skiU  at  the  end  of  ATP  for  MOS  IIH  trainees. 

Baoikaround 

Each  MDS  requires  target  tracking  skills  to  varying  degrees.  Biis 
project  looiked  at  the  continuous  tracking  b^viors  —  i.e.,  the  tracking  of 
a  target  from  the  start  of  a  simulated  missile  launch  throu^  the  simulated 
hitting  of  the  target  -  required  of  llH's  ly  the  M-70  simulator  of  the 
ground  mount  TOW?  laundier.  Bie  M-70  simulator  score  captures  the  likelihood 
that  a  soldier's  continuous  tracking  behavior  will  result  in  his  hitting  a 
moving  target,  m  this  project,  soldiers  were  tested  on  the  Project  A 
battery  when  they  entered  training  at  Port  Banning.  Their  M-70  scores  were 
obtained  from  official  records  after  they  graduated  fron  AIT  (n  =  355) . 
Aiditional  data,  including  QO/NO  GO  scores  on  8  other  IIH  tasks,  such  as  M-70 
simulator  assatibly  and  range  finding  skiUs,  were  collected  and  analyzed  for 
a  subsanple  of  81  soldiers. 

Restalts 

The  Project  A  psychcmotor  ocnposite  —  oonprising  1-handed  and  2-handed 
ta;dcs  —  predicted  !t-70  scores  for  345  soldiers  passing  ATT, 

£  -  .25.  The  graph  sumnarizes  the  extent  that  the  Project  A  psychototor 
ocnposite  iiiproves  prediction  of  TCW  gunnery  tracking  performance  over  that 
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j  pcedicted  ty  the  AFQr—a  standard  measure  of  trainahility.  (Ihe  APQT 

categories  I  and  II  represent  hic^ier  scores  and  categories  IIIB-IVA  represent 
lower  scores. )  !Ihe  APQJ  does  significantly  predict  TCW  gunnery  tracking, 

£  «  .12;  however,  the  power  of  the  Project  A  psydionotar  oonposite  over  the 
APQT  in  predicting  M-70  tracking  performance  is  diown  ty: 

o  Ihe  Project  A  Tracking  Oatposite  sorting  out  the 
better  M-70  soorans  within  relatively  honogeneous 
groipings  of  soldiers  defined  ty  the  AFQT; 
o  The  Project  A  Tracking  Oaiposite  identifying  those 
eoldift-rg  in  lower  APQT  categories  v*d  would  have 
hi^er  M-70  scores  than  soldiers  in  the  hi^ier  APQT 
categories; 

o  The  top  third  of  scorers  on  the  Project  A  Tracking 
Ocnnposite  have,  on  the  average,  M-70  scores  that  are 
21  points  above  the  average  M-70  sccsce;  a  similar  cut 
based  on  the  APQT  results  in  M-70  scores  that  are 
only  8  points  above  the  average  M-70  score. 

Project  A  tests  also  predicted  the  detailed  training  data  for  the 
subsanple  of  llH's  (n  =  81).  The  psychomotor  oatposite  had  significant 
correlations  with  M-70  scores,  r  =  .43,  and  with  a  composite  of  Number  of 
Attempts  to  achieve  a  GO  on  ei^t  AIT  tasks  r  =  .32.  (The  tasks  include 
important  IIH  tasks,  such  as  assembling  the  TCM2,  perfonning  pre-opetational 
checks  on  the  TCIM2,  and  using  an  anti-armor  range  card) .  Project  A  spatial 
tests  —  Orientation  and  Maze  —  were  the  most  consistent  predictors  of  M-70 
perfooBnoe  for  this  subsanple.  Specifically,  scores  on  Orientation  sig¬ 
nificantly  predict  M-70  tracking,  r  =  .33,  and  the  Number  of  Attempts 


Oonposite,  £  -  .34,  and  the  Maze  Test's  significant  oorrelatiQns  for  these 
two  criterion  neasures  were,  jc  =  .32,  and  ,r  =  .38,  respectively. 

Obese  results  suggest  that  M-70  tracking  performance  and  the  other  M-70 

tasks  require,  at  least,  these  dciUs: 

o  Psychorotor  —  1-handed  and  2-handed  coordination  for 
moving  a  marker  to  meet  and  remain  on  top  of  a  target 

moving  along  a  predefined  path, 
o  —  quiokly  scan  for  a  target;  determine  its 

route;  and,  perhaps,  maintain  the  target's  referent 
point  and  maintain  the  reoriented  position  of  the 
as  it  approaohes  the  target. 

Bb  m.Ject  A  laiget  Tracking,  Orientation,  and  Bare  Tests  thus  ^pear  to 
reliably  o^ture  ijrjortant  eatponents  of  IIH  tasks  and  appear  to  pccwide 

power  to  the  current  selecticn  process  of  choosing  soldiers  v*o 

vjould  betto:  perform  those  tasks. 


MEAN  M-70  SCORES  FOR  11H’s 
CLASSIFIED  BY  AFQT  AND 
PROJECT  A  TRACKING  MEASURES 
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Exploration  of  Validities  of  Project  A  Spatial  Tests 


The  Joint-Service  Selection  and  Classification  Working  Group  (JSSCWG) 
provides  technical  and  policy  support  to  the  Manpower  Accession  Policy  (M&P) 
Steering  Committee  on  matters  related  to  ASVAB  and  other  selection  and 
classification  issues.  The  Steering  Committee  is  chaired  by  the  Director  for 
Accession  Policy,  Office  of  the  Assistant  Secretary  of  Defense  (Force  Manage¬ 
ment  and  Personnel)  and  is  comprised  of  a  senior  military  officer  from  the 
offices  of  the  respective  Service  Deputy  Chiefs  of  Staff  for  Personnel,  and 
the  Commander  of  the  U.S.  Military  Entrance  Processing  Command  (USMEPCOM). 

At  the  August,  1986  meeting  of  the  Technical  Task  Group  of  the  JSSCWG,  each 
Service  representative  was  requested  to  submit  a  candidate  test  of  spatial 
ability  at  the  November,  1986  JSSCWG  meeting. 

The  technical  representatives  expressed  special  interest  in  the  Army’s 
recommendation  because  of  the  development  of  six  new  spatial  tests  for  the 
Army’s  Project  A:  Improving  the  Selection,  Classification  and  Utilization  of 
Army  Enlisted  Personnel.  Because  considerable  data  had  been  collected  on 
these  tests  during  the  concurrent  validation  phase  of  Project  A,  an  informed 
decision  concerning  the  selection  of  a  spatial  test  could  be  made  on  the 
basis  of  the  Project  A  data.  The  purpose  of  this  report  is  to  present  the 
results  of  the  validity  analyses  of  the  spatial  tests  in  the  nine  MOS  for 
which  hands-on  criterion  measures  had  been  developed. 

METHOD 

Subjects 

A  total  of  5,196  first-term' soldiers  in  nine  military  occupational 
specialties  (MOS)  provided  the  basis  for  the  results  of  this  report.  Table  1 
presents  the  nine  MOS  and  the  number  of  soldiers  tested  in  each  MOS. 


Table  1 


The  Nine  MOS  and  Sample  Sizes  Upon  Which  the  Six  Spatial  Tests  were 
Analyzed 


MOS 

DESCRIPTION 

N 

IIB 

INFANTRYMAN 

694 

13B 

CANNON  CREWMEMBER 

653 

19E 

M48  -  M60  ARMOR  CREWMAN 

499 

31C 

SINGLE  CHANNEL  RADIO  OPERATOR 

357 

63B 

LIGHT-WHEELED  VEHICLE  MECHANIC 

630 

64C 

MOTOR  TRANSPORT  OPERATOR 

678 

71L 

ADMINISTRATIVE  SPECIALIST 

504 

91A 

MEDICAL  SPECIALIST 

492 

95B 

MILITARY  POLICE 

689 

Males  comprised  88  percent  of  the  sample.  A  total  of  68  percent  of  the 
sample  were  white,  24  percent  were  black,  three  percent  were  Hispanic,  four 
percent  "other"  and  one  percent  indicated  no  racial  origin. 

Spatial  Tests 

Table  2  summarizes  the  psychometric  characteristics  of  the  six  spatial 
tests.  A  brief  verbal  description  of  each  test  follows: 

Assembling  Objects.  Each  item  requires  the  respondent  to  determine  how  an 
object  will  look  when  its  parts  are  put  together.  Some  items  have  the  parts 
labelled  with  letters  and  require  the  respondent  to  match  the  letters,  so 
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that  the  parts  can  be  put  together*  Other  items  require  the  subject  to 
mentally  put  shapes  together  to  form  one  overall  geometric  shape. 

Table  2 

Summary  Inf ormatloii^  for  the  Six  Spatial  Testa  Baaed  in  tba  Concurrent  Validation 
Phase  of  Project  A 


Test  Name 

Construct 

No.  of 

Items 

Testing 

Time 

Split-Half 

Reliabilities*’ 

Teat-Retest 

Reliability® 

Assembling  Objects 

Spatial  Visualization 

32 

16  Min 

.91 

.70 

Object  Rotation 

Spatial  Visualization 

90 

7.5  Min 

.99 

.72 

Maze  Test 

Spatial  Visualization 

24 

5.5  Min 

.90 

.70 

Orientation  Test 

Spatial  Orientation 

24 

10  Min 

.89 

.70 

Map  Test 

Spatial  Orientation 

20 

12  Min 

.90 

.78 

Reasoning  Test 

Induction 

30 

12  Min 

.87 

.65 

^Information  in  this  table  is  taken  from  Toquam,  et  al  (1986) 

Odd-even  procedure  with  Spearman-Brown  correction  based  on  Sample  of  9332  to  9345. 
^Based  on  samples  sizes  of  468  to  487. 


Qbiect  Rotation.  Each  item  consists  of  a  test  object  and  five  response 
items.  For  each  response  item,  the  respondent  must  determine  whether  the 
response  item  matches  the  test  item  or  not. 

Maze  Test.  Each  item  is  a  rectangular  maze  with  four  labelled  entrance 
points  and  four  exit  points.  The  task  is  to  determine  which  of  the  four 
entrances  leads  through  a  pathway  of  the  maze  to  one  of  the  exit  points. 
Orientation  Test.  Each  item  contains  a  non-upright  scene  within  a  circular 
or  rectangular  frame.  The  frame  has  a  circle  with  a  dot  inside  of  it.  The 
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dot  is  placed  inside  the  circle  and  at  a  point  along  the  border  of  the 
circle.  The  task  is  to  mentally  rotate  the  frame  to  align  the  circle  with 
the  bottom  of  the  scene.  After  doing  so,  the  respondent  must  determine  the 
new  location  of  the  dot  inside  the  circle. 

Map  Test.  Each  item  consists  of  a  map  that  includes  four  landmarks  spatially 
arranged  to  form  a  square.  Within  each  item,  the  respondent  is  provided 
compass  directions  by  indicating  the  direction  of  one  landmark  to  another, 
for  example,  "the  forest  is  North  of  the  campsite."  Respondents  are  also 
informed  of  their  location  relative  to  another  landmark.  Given  this 
information,  the  respondent  is  to  determine  the  direction  of  travel  between 
two  of  the  landmarks. 

Spatial  Reasoning  Test.  Each  item  consists  of  a  series  of  four  figures.  The 
task  is  to  identify  the  pattern  or  relationship  among  the  four  figures  and 
then  to  identify  among  five  possible  answers  the  one  figure  that  should 
appear  next  in  the  series. 

Armed  Forces  Qualification  Test  (AFOT).  The  AFQT  is  a  composite  score  of 
four  ASVAB  subtests  -  Arithmetic  Reasoning,  Word  Knowledge,  Paragraph 
Comprehension,  and  Number  Operations.  The  AFQT  is  used  to  classify  soldiers 
and  is  included  in  the  present  report  as  a  benchmark  to  provide  an  estimate 
of  general  ability  level.  The  reliability  of  the  AFQT  is  .92  or  .93, 
depending  on  the  ASVAB  form  that  is  used  (Palmer,  Hartke,  Ree,  Welsh,  and 
Valentine,  1988). 

Performance  Criteria 

Hands-on.  Task  Proficiency  Tests.  For  each  of  the  nine  MOS,  a  set  of  15 
critical  tasks  of  the  task  domain  were  selected  for  hands-on  testing.  Each 
task  comprised  several  performance  steps  and  each  step  was  scored  pass  or 
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fail.  A  proportion-passed  score  was  derived  for  each  task  and  the  proportions 
were  averaged  across  tasks  to  yield  an  overall  hands-on  test  score  for  each 
soldier. 

School  Knowledge  Tests  (K3K  These  written  tests  were  developed  to  represent 
the  instructional  material  developed  within  the  Advanced  Individual  Training 
course  for  each  of  the  nine  MOS.  Each  test  consists  of  100  to  200  written 
items  and  covers  30  tasks. 

Job  Knowledge  Tests  (K5).  These  written  tests  consist  of  three  to  fifteen 
questions  on  each  of  a  sample  of  30  representative  tasks  and  include  the  15 
tasks  considered  critical  for  hands-on  testing.  The  K5  tests  consist  of  100 
to  200  written  job  knowledge  items. 

In  summary,  the  six  Project  A  tests  will  be  validated  against  job 
performance  criteria  for  nine  MOS.  To  the  extent  that  the  Project  A  spatial 
tests  measure  different  abilities  or  skills,  one  may  ask  which  skill  best 
predicts  performance  and  what  are  the  unique  characteristics  of  the  test 
representing  the  skill.  The  focus  of  this  paper  is  to  concentrate  on  the 
unique  characteristics  of  each  spatial  test  and  based  on  those 
characteristics  decide  on  the  "best"  spatial  ability  test  for  the  Army. 

RESULTS 

Table  3  presents  the  means  and  standard  deviations  for  the  Armed  Forces 
Qualification  Test  (AFQT)  and  the  Project  A  spatial  tests  for  the  nine  MOS. 
Again,  the  AFQT  is  used  to  classify  soldiers  and  is  included  as  a  benchmark 
to  provide  an  estimate  of  general  ability  level.  One  would  want  to  determine 
if  this  estimate  of  ability  level  corresponds  with  ability  as  defined  by  the 
other  measures. 
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TABLE  3 


Means  and  Standard  Deviations  for  AFQT*  and  Mechanical  Comprehension  (MC) 
Scores  from 

ASVAB  and  Project  A  Spatial  Measures  for  Nine  MOS 


MOS  n 


Assembling 

AFQT^  Objects^  Map  Maze 


Orient-  Object  Spatial 

ation  Rotation  Reasoning 


IIB 

(439) 

56.3 

(22.0) 

24.6 

(6.1) 

9.6 

(5.5) 

13B 

(420) 

47.0 

(20.1) 

22.7 

(6.5) 

7.0 

(5.4) 

19E 

(357) 

53.8 

(21.6) 

24.3 

(5.9) 

9.1 

(5.7) 

3 1C 

(256) 

59.1 

(19.8) 

24.2 

(5.9) 

8.5 

(5.4) 

63B 

(429) 

49.6 

(19.3) 

24.6 

(6.0) 

8.1 

(5.5) 

64C 

(479) 

42.9 

(18.4) 

23.0 

(6.3) 

7.0 

(5.2) 

71L 

(389) 

57.0 

(19.3) 

23.2 

(6.6) 

6.8 

(5.1) 

91A 

(340) 

61.0 

(17.9) 

25.8 

(5.2) 

8.9 

(5.4) 

95B 

(499) 

62.4 

(17.0) 

25.2 

(5.3) 

10.2 

(5.4) 

17.6 

(4.3) 

12.0 

(6.2) 

64.9 

(18.9) 

19.8 

(5.7) 

16.2 

(4.8) 

10.1 

(5.8) 

60.5 

(19.5) 

18.4 

(5.6) 

17.4 

(4.5) 

11.8 

(6.3) 

64.3 

(18.7) 

20.3 

(5.5) 

17.3 

(4.4) 

11.9 

(6.3) 

64.3 

(19.4) 

20.5 

(4.5) 

17.2 

(4.4) 

12.0 

(6.3) 

64.5 

(19.0) 

19.7 

(5.5) 

16.3 

(4.5) 

10.3 

(5.8) 

62.5 

(18.2) 

18.4 

(5.5) 

15.4 

(4.6) 

10.4 

(5.8) 

58.3 

(17.8) 

19.5 

(5.2) 

17.0 

(4.6) 

12.8 

(6.2) 

64.5 

(17.3) 

21.2 

(4.4) 

18.2 

(3.9) 

13.0 

(6.0) 

69.9 

(15.6) 

21.2 

(4.3) 

No^.  ASVAB  =  Armed  Services  Vocational  Aptitude  Battery  (See  Text). 
AFQT  ■=  Armed  Forces  Qualification  Test  Composite  (See  Text). 

The  AFQT  is  a  percentile  score. 

Assembling  Objects  and  the  other  spatial  measures  are  number  correct, 


The  13B  soldiers  have  the  lowest  average  score  on  all  the  Project  A 
spatial  measures,  except  for  the  Maze  Test,  while  the  95B  soldiers  are 
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consi.st6n'kly  hi.gh6st  on  &11  spstial  moasures.  The  13B  have  the  second  lowest 
average  AFQT  and  the  95B  have  the  highest.  These  descriptive  statistics 
indicate  that  the  average  spatial  ability  differences  among  the  MOS  coincide 
with  the  average  AFQT  differences  among  the  MOS,  and  that  in  general,  the 
Project  A  spatial  measures  are  most  likely  affected  by  AFQT  differences. 

Table  4  presents  the  means  and  standard  deviations  for  each  MOS  for  the 
school  knowledge  (K3),  job  knowledge  (K5),  and  hands-on  measures.  The  K3,  K5 
and  hands-on  test  scores  are  raw  scores  (i.e.,  percent  correct).  Table  4 
also  presents  descriptive  statistics  for  the  Core  Technical  and  General 
Soldiering  tasks.  The  Core  Technical  tasks  are  composed  of  tasks  unique  to 
the  MOS.  The  General  Soldiering  tasks  are  composed  of  tasks  that  are 
required  of  all  soldiers,  such  as  administering  CPR,  putting  on  clothing  for 
Nuclear  Biological  Conflict  (NBC),  and  preparing  an  M-16A  rifle  for  firing. 
The  Core  Technical  and  General  Soldiering  scores  are  standardized  scores  that 
are  a  composite  of  K3,  K5,  and  hands-on  tests,  as  well  as  ratings  of  soldier 
performance . 

Variables  accounting  for  the  average  differences  across  the  MOS  hands-on 
and  knowledge  scores  may  include  a  number  of  characteristics  related  to 
soldier  training.  These  characteristics  may  include  (a)  the  Project  A  tasks 
are  easier  than  other  tasks  in  which  soldiers  are  trained,  (b)  Project  A 
tasks  reflect  the  relatively  lower  sophistication  required  for  some  tasks  in 
some  MOS,  or  (c)  Project  A  tasks  represent  tasks  that  are  given  much  more 
practice,  regardlessof  their  relative  sophistication,  with  some  MOS  given 
more  practice  on  tasks  than  are  other  MOS. 
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Table  4 


Means  and  Standard  Deviations  for  School  (K3).  Job  Knowledge  (K5). 
Hands-on  (HO),  Core  Technical  and  General  Task  Criterion  Measures 
for  Nine  MOS 


CRITERION  MEASURES 


MOS: 

K3 

IIB 

62.1 

(11.4) 

13B 

56.5 

(10.3) 

19E 

68.7 

(10.8) 

31C 

60.3 

(11.4) 

63B 

61.0 

(12.3) 

64C 

63.1 

(10.6) 

71L 

59.2 

(10.6) 

91A 

60.3 

(10.3) 

95B 

59.4 

(9.6) 

K5 

HO 

61.9 

(10.5) 

70.4 

(7.4) 

64.1 

(10.2) 

62.4 

(11.3) 

63.3 

(9.2) 

77.8 

(7.3) 

60.6 

(9.9) 

70.4 

(7.8) 

66.2 

(10.0) 

84.7 

(4.8) 

59.6 

(9.4) 

71.6 

(7.6) 

58.2 

(9.9) 

59.3 

(8.2) 

66.9 

(9.2) 

72.5 

(7.0) 

62.9 

(8.7) 

70.4 

(6.1) 

CORE 

TECH 

GENERAL 

TASK 

102.9 

(16.1) 

102.9 

(16.1) 

102.0 

(16.9) 

102.7 

(15.4) 

102.7 

(15.0) 

102.5 

(15.7) 

101.5 

(17.0) 

101.8 

(16.6) 

102.6 

(15.2) 

102.3 

(14.9) 

102.0 

(14.5) 

101.9 

(16.3) 

101.2 

(17.4) 

101.3 

(16.7) 

102.9 

(15.9) 

102.2 

(16.4) 

100.9 

(14.7) 

101.1 

(16.1) 

*Core  technical  and  general  task  are  the  same  items  for  IIB. 


Correlations  of  spatial  ability  measures  with  hands-on  and  other 
criterion  measures  may  reflect  the  extent  that  these  abilities  predict  job 
performance,  despite  the  inherent  variations  across  MOS.  Where  prediction  of 
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a  criterion  measure  (or  measures)  is  consistent  across  MOS,  one  may  ask, 

"What  common  characteristic  is  being  predicted  and  how  important  is  that 
characteristic  for  job  competence? • . 

Tables  5  and  6  provide  the  correlations  of  the  Project  A  spatial  measures 
and  the  AFQT  with  the  criterion  measures.  The  correlations  are  the  median 
correlations  of  each  test  with  criterion  measures  across  the  nine  MOS.  The 
range  of  the  correlations  across  the  nine  MOS  are  in  parentheses.  Table  5 
contains  correlations  uncorrected  for  range  restriction  and  Table  6  contains 
correlations  corrected  for  range  restriction.  Table  5  answers  the  question, 
"Given  the  current  system  for  selecting  and  classifying  soldiers  for  the 
^iffsrsnt  MOS,  which  spatial  test  is  the  more  consistent  predictor  of 
performance?".  Table  6  answers  the  question,  "Given  the  possibility  of 
changing  the  selection  and  classification  system  and  making  the  maximal  use 
in  using  the  spatial  test  for  selecting  and  classifying  soldiers,  which 
spatial  test  could  become  the  most  consistent  spatial  predictor  of 
performance?" 

Table  5  shows  that,  even  with  a  group  of  soldiers  that  are  selected  for 
the  MOS  and  have  stayed  in  the  Army  for  18-24  months,  the  Project  A  tests 
have  moderate  correlations  with  all  the  criterion  measures ,  except  for  the 
hands-on  measures.  The  AFQT  is  a  similarly  reasonable  predictor.  Table  6 
shows  that  correcting  for  range  restriction  results  in  a  change  in  the 
relative  magnitude  of  the  correlations  of  the  spatial  tests  with  the 
criterion  measures.  For  example,  the  Map  Test’s  median  correlation  with  the 
K3  tests  is  22  percent  (r  -  .61  vs.  r  =  .50)  larger  than  the  correlation 
of  the  Assembling  Objects  Test’s  median  correlation  when  correcting  for  range 
restriction.  When  one  does  not  correct  for  range  restriction,  the  Map  Test’s 


9 


median  correlation  is  only  14  percent  (r.  =■  .40  vs.  r_  =  .35)  larger  than  the 
Assembling  Object's  median  correlation. 


Table  5 

Median  and  Range  of  Uncorrected  Correlations  Between  Each  Spatial  Test  and 
AFQT  with  School  (K3).  Job  Knowledge  (K5).  Hands-On  (HO).  Core  Technical  and 
General  Task  Criterion  Measures  for  the  Nine  MOS 


CRITERION  MEASURES 


K3 

K5 

HO 

CORE 

TECH 

GENERAL 

TASK 

ASSEMBLING  OBJECTS 

35 

(28-41) 

43 

(35-50) 

20 

(16-33) 

34 

(23-27) 

37 

(31-45) 

MAP 

40 

(32-54) 

44 

(32-59) 

24 

(15-34) 

31 

(26-47) 

38 

(34-51) 

MAZE 

21 

(15-26) 

29 

(20-33) 

16 

(15-27) 

21 

(10-25) 

25 

(15-30) 

OBJECT  ROTATION 

17 

(04-25) 

22 

(13-31) 

19 

(10-28) 

19 

(07-30) 

23 

(19-29) 

ORIENTATION 

31 

(18-37) 

35 

(27-43) 

22 

(15-33) 

28 

(19-38) 

33 

(28-39) 

SPATIAL  REASONING 

34 

(27-43) 

40 

(36-51) 

27 

(13-38) 

33 

(23-44) 

38 

(30-46) 

AFQT 

40 

(31-52) 

40 

(33-52) 

21 

(08-33) 

35 

(23-44) 

37 

(30-45) 

This  differential  effect  on  the  correlations  when  correcting  for  range 
restriction  also  results  in  a  change  in  the  rank  of  the  spatial  tests  ,  when 
ranking  them  from  best  to  worst  predictors.  For  example,  when  correcting  for 
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range  restriction, 
K3  tests  than  does 


the  Spatial  Reasoning  Test  has  a  higher  correlation  with 
the  Assembling  Objects  Test;  without  correcting  for  range 


TABLE  6 

Median  and  Range  of  Corrected  Correlations  Between  Each  Spatial  Test  and  AFQT 
with  School  (K3).  Job  Knowledge  (K5).  Hands-On  (HO).  Core  Technical  and 
General  Task  Criterion  Measures  for  the  Nine  MOS 


K3 

K5 

ASSEMBLING  OBJECTS 

50 

58 

(43-56) 

(48-61) 

MAP 

61 

64 

(56-68) 

(53-73) 

MAZE 

37 

46 

(32-49) 

(39-51) 

OBJECT  ROTATION 

35 

37 

(27-46) 

(31-46) 

ORIENTATION 

48 

53 

(42-56) 

(43-58) 

SPATIAL  REASONING 

57 

60 

(42-60) 

(49-65) 

AFQT 

67 

66 

(51-75) 

(54-71) 

CRITERION  MEASURES 


HO 

CORE 

TECH 

GENERAL 

TASK 

34 

(20-44) 

46 

(35-51) 

52 

(44-55) 

41 

(20-53) 

53 

(38-65) 

58 

(51-68) 

32 

(17-43) 

38 

(29-48) 

42 

(36-51) 

27 

(15-36) 

35 

(22-42) 

38 

(35-42) 

36 

(16-46) 

45 

(32-53) 

49 

(40-57) 

44 

(18-52) 

52 

(31-60) 

56 

(43-64) 

42 

(10-62) 

61 

(33-71) 

63 

(48-66) 

restriction,  the  Assembling  Objects  Test  has  the  higher  correlation.  A 
general  statement  would  be  that  the  tests  with  the  largest  validities,  in 
order  from  highest  to  lowest,  appear  to  be  the  Map,  Spatial  Reasoning, 
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Assembling  Objects,  and  Orientation  tests.  Again,  the  AFQT  appears  to  be  a 
very  good  predictor  of  performance. 

Two  other  criteria  for  determining  the  "best"  spatial  test  may  include 
(a)  the  extent  of  convergence  of  the  Project  A  spatial  tests  with  a 
previously  used  ASVAB  spatial  test  and  (b)  the  extent  of  divergence  of  the 
Project  A  spatial  tests  from  the  AFQT. 

The  ASVAB  5/6/7  spatial  test  was  called  the  Space  Perception  Test.  The 
test  required  that  a  respondent  identify  a  three-dimensional  figure  obtained 
from  folding  a  flat  figure  (Maier  and  Grafton,  1981).  It  had  20  items  and 
required  12  minutes  for  administration.  Test  -  retest  reliabilities  were  .71 
(Bayroff  and  Fuchs,  1970).  Table  7  shows  that  the  Space  Perception  Test 
correlates  highest  with  the  Assembling  Objects  Test,  r  =  .41,  and  lowest  with 
the  Map  and  Spatial  Reasoning  Tests  r.  =  .28  (N  =•  431).  To  the  extent  that 
the  Space  Perception  Test  represents  an  important  ability  predictive  of 
military  performance,  one  may  state  that  the  Assembling  Objects  Test  has  more 
in  common  with  that  ability  than  the  other  Project  A  measures. 

Alternatively,  Table  7  shows  that  the  Map  Test  has  the  highest 
correlations  with  the  current  AFQT,  with  the  correlations  between  Assembling 
Objects  and  the  AFQT  being  much  lower.  To  the  extent  that  the  current  AFQT 
is  used  as  a  standard  for  placing  a  soldier  on  a  scale  of  trainability ,  one 
sees  that  Assembling  Objects  could  potentially  provide  more  information 
toward  making  that  scale  a  better,  more  comprehensive  measure  of  trainability 
than  would  the  Map  or  the  Spatial  Reasoning  Test. 
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Table  7 


Correlations  of  ASVAB  5/6/7  Space  Perception  Test  and  the  AFQT  with 
Project  A  Spatial  Tests 


ASVAB  5/6/7 

AFOT 

PROJECT  A  TESTS 

SPACE  PERCEPTION 

MEDIAN 

RANGE 

ASSEMBLING  OBJECTS 

.41 

.34 

.30- 

.39 

MAP 

.28 

.50 

.46- 

.62 

MAZE 

.37 

.30 

.23- 

.34 

OBJECT  ROTATION 

.26 

.23 

.18- 

.30 

ORIENTATION 

.31 

.38 

.33- 

.44 

SPATIAL  REASONING 

.28 

.48 

.40- 

.52 

Note.  N  =  431 

®These  correlations  represent  the  median  correlation  and  range  of 
correlations  of  the  AFQT  with  the  Project  A  tests  across  the  nine  MOS. 
Sample  sizes  were  too  small  for  some  MOS  for  computing  the  median 
correlation  with  the  ASVAB  5/6/7  Space  Perception  Test. 


CONCLUSION 

One  may  conclude  that  the  Project  A  spatial  tests  correlate  with  job 
performance  reasonably  well,  although  the  spatial  tests*  correlations  with 
hands-on  performance  appear  to  be  relatively  lower  than  with  the  other 
criterion  measures.  Four  spatial  tests  appear  to  have  good  criterion 
validities,  with  the  Map  Test  showing  the  strongest  validities  across  the 
criterion  measures.  However,  the  Assembling  Objects  Test,  which  shows 
reasonable,  although  not  as  strong  validities  as  the  Map  Test,  does  show  a 
much  higher  convergence  with  a  previous  ASVAB  spatial  test.  The  Assembling 
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Objects  Test  also  shows  much  more  divergence  from  the  AFQT,  a  composite  of 
four  ASVAB  tests  that  is  currently  used  to  represent  trainability  potential. 
Given  these  combination  of  factors,  it  is  concluded  that  the  Assembling 
Objects  Test  shows  more  promise  as  a  test  that  could  add  most  toward 
constructing  a  measure  of  trainability  (i.e.,  AFQT)  that  includes  a  spatial 
test. 
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Abstract 


Technique,  for  e.tlnatlnh  the  duller  benefit  of  Incree.ed  level,  of 
,roductl,lt,  or  Inpruved  perfur-nce  cell  for  ..tlnetlon  of  the  duller 
vein,  uf  p.rfur»nc.  et  .l«n  level.  >0^  of  S0$.  the  .tend.rd  devletlun  uf 
perfumence  quellty  «..ured  In  duller..  Such  ..tlnete.  have  been  obtained 
vie  coat  eccuuntln*  or  by  .upervl.ur  eetlnet...  Thl.  re.eerch  ettenpt.  to 
e.tl«t.  utility  fur  verluu.  perfutnence  level.  In  the  context  uf 
expenelve.  cunplex  «epun.  .y.t«..  Chengee  In  ey.ten  perfutnence  ere 
dl.cu..ed.  here.  In  terns  uf  change.  In  the  nunber  uf  .y.te.  unit,  required 
fur  e  fixed  level  uf  eccunpllshnent.  Fumulee  ere  presented  fur  llnhlng 
these  changes  to  change.  In  the  utility  of  the  .y.te.  In  dollar., 
infumatlun  about  .y.te.  perfumence  changes  can  also  be  entered  Into 
nation,  to  cunpute  SOS.  These  techniques  fur  estlnatlng  the  vslue  uf 
peclumance  level,  end  decerning  SOS  my  be  useful  eddltlun.  to  traditional 
neat  accounting  and  nemr  auparvl.ury  ..tlnatlun  technique.. 
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Alternate 


Method,  of  E.ttotlng  the  DolUr  V.loe  of  F«f.r«oc. 


T..tlog  to  loprov.  ..lection  .nd  cU..lflc.tlon  decl.lon.  1.  . 
not..!  P.tt  of  ..t.rlng  into  »plof«.t  1.  «>«  ord-ntctlon.  .nd 

in  nanp  -U  one.  (Prledoan  d  Wllll»..  The  benefit,  of  »ch 

t..tln8  h.,.  been  ,n..tloned  on  .n  lncr...ln8ly  frequent  he.!.,  both 
It.  the  perspective  of  f.lrne.s  end  In  ter.  of  the  benefit,  .ccrued 
ftoo  .onetl..  costly  .election  prograss.  Co.t  benefit  enslyse.  of 
.election  end  cleselflcstlon  procedures  frequently  .tress  the  price  o 
recruiting  end  testing.  Uhll.  neglecting  the  doll.r  benefits  of 
resulting  incresses  In  productivity.  This  oversight  lesves  .neg.rs 
eeehle  to  ..t.n.te  the  dollsr  velue  to  the  org.nls.tlon  of  .election  end 

classification  procedures. 

The  utility  of  Inplenentlng  v.rlou.  personnel  policies  Is  loport.nt 
dec.  for  nsnsgcnt  declslon.aUng.  The  costs  of  l.pl...ntl«l 
personnel  policies  .st  be  weighed  eg.lnst  entlclp.t.d  benefits. 
I^le^ntstlon  costs  ere  ususlly  couched  In  reel  duller  tems.  Eesl  y 

estlnsted  costs  ere  essocl.ted  with  ..Urles,  .P— 

ednlnlstrstlon.  fees,  per  die.  p.ld  to  eppllcsnts.  conput.r  tine  end 

n-npflts  accrued  from  implementation  of 
personnel  for  scoring,  etc.  Benefits  accru 

..■«  not  as  clearly  identifiable  In  dollar 
personnel  policies,  however,  are  not  as 

terns,  dudgnents  of  the  net  positive  l.p.ct  of  Inplenentlng  given 
personnel  policies  ere.  therefore,  difficult  to  nsbe. 
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3 

E.rl,  .ppr..ch«  to  th.  bonoflt.  ...ocUtod  «ltb  totting 

„.gt».  lotuted  on  the  cott.lotlon  botnoon  grodlctor 
.rltorlo.  porfomonto.  KoHy  (1«3)  d.flnod  th.  -Indot  of  foroctl.* 

Efflclonc-  ..  It.l-  JH?-  * 

goto^lnotlon-  «.  ptopoood  by  Hull  (1928).  Ihl.  Indox  i.  .Lply  I  • 

Ob.  proportion  of  trlt.rlon  oorUnc.  octonntod  for  by  the  prodlctor 
«..ur...  .otb  .pproocho.  .uggdot  tbot  only  touting  progranu  with  blgb 

validity  coefficients  are  beneficial. 

Hovovor.  inportant  -nagonont  Infomatlon  1.  nogloct.d  by 

„tha.atl=al  procoduro.  Involving  only  validity  or  roUtlv.  varUnco. 

Taylor  and  Ruao.ll  (1939)  Incorporatod  Infomatlon  .bout  o.loctlon 

ratio,,  th.  proportion  of  applicant,  ooloctod  to  tbo.o  «bo  apply,  .nd 

infomatlon  auout  tb.  baa.  rat.  of  auccaful  crlt.rlon  porfomanc 

Tbolr  conputatlon,  dlcboto.lt.  crlt.rlon  porfomanc.  Into  ..tlaf.ctor, 

and  unaatlafactory.  and  cloarly  abo.  that  Itb  anfflclontly  mall 

..loctlon  ratio,,  man  to.ta  «ltb  low  valldltlo.  can  Imd  to  Inportant 

incroa,..  In  tb.  nunbor.  of  .ucc.ful  pcrfomor,.  Incr.a...  In  tb. 

proportion  of  .ucc...fnl  porfomcr,  pr.dlct.d  by  tb.  ..loctlon 

,  .n  b.  llnkod  to  th.  lncr..,.d  nuofuln...  or  utility  of  an 
instrument  can  be  linttea  to 

instrument. 

Htogd.n  (1949)  and  Cronbach  and  Gl.a.r  (1965)  alto  addr.,, 
pplllty  ..tlmtlon,  Tb.lr  fomulatlon.  doalt  Itb  contlnuou.  l.v.1.  of 

erltorlon  porfomanc.  r.tbor  than  tb.  dlcbotony  ua.d  by  I.ylnr  «.d 
.....11  (1939).  Tboy  llnbod  nomally  dlatrlbutod  porfomanc.  l.v.1.  to 

tb.  val...  ..tlnatod  for  tbo..  porfomanc.  l.v.1..  m  dolUr.. 
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..in  i.  P«iucti.it,.  ..  uiiUt,  (»y.  .M.i..-  ^  -‘"S 
,.lld  ..Uctlo.  procure,  include.  (.)  S».  indiriCl. 

selected  (M  §£1,  the  .tenderd  devl.tlon  of  perforMUce,  .c 
..Hit,  «tric  euch  ..  dolUre,  end  (c)  the  .u.r...  perfomenc.  etpect. 

„  the  criterion  h,  the  .elected  .roup  ..  ..tineted  iron  .  mlid 

predictor,  given  by 

PS  -  Ms  SDS  Rxy  Zx 

TO  eccount  for  the  co.t  (Ct)  of  t.etin.  e.ch  of  K.  ePPUcnt.  the 

formula  was  adjusted  to 

Tig  »  Ss  SDS  Rxy  Zx  -  Na  Ct 

A  .or.  couplet,  deecription  of  .uch  for.ul.tlon.  cn  he  fouud  in 
Croubech  .nd  Gle.er  (1965),  Bucter  .nd  Sctaldt  (1982).  nnd  C.clo 

(1982).  .AC 

«hll.  the  vein.,  of  nost  of  the  v.tiehle.  ou  the  right  hend  .  e  o 

the  grogden-Crooh.ch-Gl..er  fon»l..  .re  hnown.  the  e.tlMtloc  of  SDS. 

Ch.  etecderd  deui.tion  of  perfor^nc.  .clod  in  doller.,  i.  prohleuetic. 

A  ..cent  revie.  h.  Hunter  end  Sch.ldt  (1982)  report,  thet  oul,  t« 

u  -9.9.»«Tited  the  computation  of  SD$  using  cost 
published  efforts  have  attempted  the  c  p 

accounting  methods. 

An  alternative  to  cost  accounting  estimates  is  to  estimate  the 
..ller  uelue.  to  the  org.nlt.tio.  of  P.rfom.nc.  .t  the  50th  percentile 
leuel.  the  85th  percentile  level  (on.  .r.nd.rd  devletion  .hove  the 
«.n).  .nd  eoneti...,  the  15th  percentile  level  (on.  -tenderd  deviet  on 
h.10.  the  «.n).  The  dolUr  difference  hetueen  the  15X  .nd  50X.  C  50X 
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and  85Z  points  provides  an  eatlnate  of  SD$^.  This  SDg_Esti»ntl 

t.chal,«.-  «.  »y  C..ci.  ...d  Sllb.,  (1»79)  wlrt  ..c«.d  l.«l  , 

1.  food  ...d  b.v«.g.  (H..n  -  ?30.000.  SDl*  S>.S00)i  by 

Scb.ldt.  Hunter,  HcKentl.,  .nd  M«ldr»t  (1979)  -1th  co-put.t  progr«.m.r. 

(SDl-  910.413)!  »J 

.  sil.327);  .nd  by  bobbn,  lt.rr.n.  .nd  y.rklngton  (1983)  -Itb  ln.ur.nc. 

coun..l.r.  (Medlnn  -  896.000.  SDi  '  1"  I- 

..1.,  d.t.  -nr.  .1.0  .v.ll.bl.  .nd  yl.ldnd  ..l..-b...d  .tnti.tlc.  -hlch 

«,r.  do.,  to  tbo..  obt.ln.d  by  tb.  r.tlng  «thod  (Madlnn  -  8117.300. 


SD$  «  $52,308). 

Th.  SD8.8tlB.tlon.  r.por.d  .boy.  -.r.  derived  In  nontext,  -her. 
pertotBann.  ...  ...Uy  1"  ‘“H"-  521  ..tl.«lo» 

,ue.tlon.  developed  by  Hunter  .nd  ScbBldt  (1982)  ..bed  for  ..tlBUte.  of 
the  -V.1U.  to  tb.  egeney-  of  verlou.  p.rfor..nn.  level..  Tb.  gueetlon. 
«re  preceded  by  Inetruntlon.  to  -con.ld.r  tb.  co.t  of  bevlng  en 
outside  fltB  provide  these  product,  end  eervlce."  (Scheldt  et  .1..  1979 
for  computer  progrcer.).  or  to  -noclder  «h.t  tb.  co.t  -ould  be  of 
b.vlng  en  <«.t.ld.  consulting  firm  produce  these  product,  end  servlc..- 
(Hunter  S  Schmidt.  1982.  for  budget  .nelysts).  Both  Ce.clo  end  Sllbey 
(1979)  .nd  Bobko  et  .1  (1983)  freed  que.tlon.  In  terms  of  performance 


value  and  estimates  of  total  yearly  dollar  sales. 

It  occurred  to  u.  tb.t  there  may  be  situation,  -her.  such  ..tlmate. 
«,uld  be  impractical.  Th...  could  occur  -her.  tb.  nature  of  tb.  -ork 
such  that  eneger.  -ere  more  .ccustomed  to  considering  the  reUtlve 
productivity  of  eployee.  or  cr««  then  th.  co.t.  of  hiring 
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replacements.  These  could  also  occur  where  employees  operate  very 
complex,  expensive  equipment  and/or  are  focal  to  the  productivity  of  a 

costly  system. 

Two  methods  came  to  mind  which  might  better  serve  in  such 
circumstances.  The  first  was  somewhat  like  the  ^Estimation 
Technique.  Instead  of  using  estimates  of  the  dollar  value  of  85th 
percentile  performance,  however,  the  technique  uses  estimates  of  how 
many  superior  (85th  percentile)  performers  would  be  needed  to  produce 
the  output  of  a  fixed  number  of  average  (50th  percentile)  performers. 

This  estimate,  combined  with  an  estimate  of  the  dollar  value  of  average 
performance,  provides  an  estimate  of  and  is  the  basis  for  the  name 

"Superior  Equivalents  Technique". 

The  second  technique  is  a  slight  extension  of  the  Brogden- 
Cronbach-Gleser  formula.  The  derivation  is  based  on  a  system  of 
productive  units,  each  unit  with  two  possible  output  levels,  average  and 
superior.  Overall  system  performance  can  be  increased  by  increasing  the 
performance  of  the  units  from  average  to  superior,  or  by  increasing  the 
number  of  units  with  average  performance.  SD£  and  the  utility  of  the 
selection  system  can  be  indexed  by  the  cost  of  the  increased  number  of 
average  performing  units  required  to  achieve  the  overall  increase  in 
system  performance.  Cost  accounting  procedures  can  be  used  to  estimate 
the  cost  of  each  unit.  We  called  this  the  "System  Effectiveness  Technique" 
The  purpose  of  this  research  was  to  develop  the  "Superior 
Equivalents  Technique"  and  the  "S/stem  Effectiveness  Technique",  and  to 
apply  both  and  the  SD$  Estimation  Technique  to  an  existing  system.  We 
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cho«  .  .y.t«.  th.  .,ulp..nt  1.  c<»pl.x  .nJ  .xpen.l«  »»•« 

operator,  ona  .opervl.or.  .r.  f«  wro  .cco.to.ed  to  thlAl«g  .bout  th. 
0.10.  of  ,.rf.r«nt.  l.v.1.  In  t.r«  of  op.r.tl.n.l  output  t.th.r  .h.n 
domt  «lu..  ThI.  .,.t-  1.  cooprued  of  th.  t.uh.  .nd  th.lr  or...  1. 

the  U.S.  Army. 


Method 


First,  th.  Superior  E,ulv.l.nt.  .nd  S,.t«..  Eff.ctt«n... 

I.chul,u..  «r.  d.v.lop»i  to  th.  «t.ut  th.t  th.,  could  b.  .PFllcd  to 
t.*  cult.,  socoud.  ..tl«t..  r.,ulr.d  h,  th.  Superior  S,ulv.l.ut. 
lechulqu.  .ud  th.  »r.  coov.otlon.l  SDl  ..tl«.tlou  t.cholqu.  «r. 

.oohlnad  into  ou.  ,u..tlonn.lr.  .nd  .dolnl.t.r.d  to  .group  of  t.uk 
coouenders.  Sat.  required  lor  th.  S,.t».  Ef fectlvenee.  T.chulqu.  «r. 

obtained  from  the  literature. 

c,.e.rlor  Euu)..lent.  Technique  Pev.lof.^ 

The  Superior  Equivalent.  Technique  1.  couc.ptu.ll,  elnllar  to  th. 

conveutloual  S£1  Eetheatlon  T.chulqu.  uhlch  ..tlmat..  SDS 

difference  In  dollar  ..tluate.  of  th.  value  of  SSth  and  50th  percentile 

perfomance.  The  haalc  concept  of  the  Superior  Equivalent.  Technique  1. 

that  of  eatinatlng  th.  .t.nd.rd  d.vl.tlon  of  p.rforwnc.  In 
perfomanc.  unit.,  and  then  converting  to  dollar  unite.  «.  ...uned  th.t 
.up.rvl.or.  are  .ccu.to«d  to  evaluating  th.  relatlv.  pcrfomance  of 

ttelr  »pl.,c...  .nd  can  «k.  accurate  Judgoent.  In  th... 

e.  U  a.  Hri-le  diffiC’ilty  In  estimating  the  number 

Accordingly,  they  should  have  little  dltticix  y 

(E85)  of  SSth  percentile  »plo,...  required  to  equal  th.  perfomance  of 
„„  fined  n-her  (SSO)  of  ev.r.g.  p.rfor«r..  Eh.re  the  vein,  of 
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,«r.8.  ,«rfor.«c.  (V50)  1.  too™,  or  c.o  U  SBi«T  f 

..tl«tad  by  0.108  the  r.tlo  of  NSOfflSS  time.  V50  to  obt.1.  VM.  ood 

tboo  ,obtr.ctfo8  V50.  »!•  «= 

SDS  ■  V85  ~  V50 

V85  -  (V50)(N50) 

K85 


Hence » 


SD$  -  (V50KN50)_  j 
H85 


(2) 


la  oor  c.  -o  ..t  2H  -  “  ‘ 

caaotodors.  boco..  tb.r.  .r.  W  t.r^  in  .  cob  coop-ny.  «.  o..d  too 

Kthod.  for  ..tlMtlog  V50.  Flr.t,  «  o..d  .n  ..tloate  baaad  on  tba 

.alary  and  banafit.  paid  tba  avaraga  tanb  co-aaodar.  In  ganaral.  ona 

night  ...ooa  organlaatlon.  pay  avaraga  aoployaa.  .boot  .hat  tb.y  art 

aaA  an  Item  on  our  questionnaire  asking  the  dollar 
worth.  Second,  we  used  an  item  on  out  h 

value  of  average  performance. 

System  Effectiveness  Technique  Development 

Tba  Syataoa  Effactlvanaa.  Tacbnlqoa  do.,  not  raly  on  aatloata.  of 
S2J.  baaad  on  dlffaranca.  to  tba  aatl^tad  valoa  of  varlooa  parforoanca 
l.«U.  loataad.  It  1.  baaad  on  tba  conc.pt  that  loprovad  parforoanca 
,ltb  an  axlatlng  nonbar  of  onlt.  <«.ploy../«acblna.)  to  a  .y.t«.  ooold 

plaid  ovarall  ayatao  ootpot  agoal  to  that  glvan  axlstlnp.  par 

,  v-y  of  units.  The  value  of  the  improved 
unit  hut  with  an  Increased  number  of  uni 

V  Mad>  of  the  increased  number  of  units.  When 
performance  would  equal  the  cost  of  the  increa 
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rt.  ,rogd«-Cronb.chH!l...r  fom.!.  1.  r.«.t  l«  th...  t.t«,  the 

rtovld..  .  uttUt,  fo™U  1.  .hlch  1.  replaced 
.ore  reedU,  obteleed  co.t  end  perfor»nc.  ter«.  Our  derlretlons 

follow: 

Let  the  coet  of  e  eingle  unit  In  e  eyeten  he  Cu.  Ut  the  overall 
.,.tm.  effectlvenea.  he  t.  thl.  «y  he  echleved  .Ith  varying  nunhers  of 
unite  depending  on  the  perfomance  of  the  unite.  Or 

(3) 

Y  ■  nlYl  ■  n2Y2  »  »  «  pIYI 

.Here  nl  -  nunher  of  unite  nt  perfomance  level  1.  on  a  ratio  ecale.  and 
Y1  ■  man  performance  of  unite  at  level  on  a  ratio 

Examplee  of  performance  ecalee  ueeahle  In  thle  formula  ere 
prohahlllty  of  hlte  per  firing  (Army  tank  commander),  number  of 
convlctlone  per  year  (detective),  nmher  of  puplle  echlevlng  given  a 
etandard  (teacher),  or  other  freguency-type  varlablee.  In  a  ayetem 
Where  Improved  performance  «  le  obtained  from  each  of  the  Initial  nl 
unite,  the  overall  Improvement  In  eyatem  performance  la 

(4) 

nlY2  -  nlYl  ■  tY2  -Yl)_ 

The  number  of  extra  unite  operating  at  the  Initial  performance 
level  which  are  needed  to  echleve  thle  improved  performance  le 
DEL  n  ■  nl  (Y2  -  Yl) 

Ike  dolUr  vain!  of  proved  performance  le  egulvele.t  to  the  extra  nmber 
of  lower  perfomlng  unite  needed  tlmee  the  coet  pet  unit 
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or 


Dfi  »  Cu  DEL  n 

U$  •  Cu  nl  (Y2  ~  VI) 

Y1  _ 


(6) 


Stapl,  the  value  In  dollers  of  .chlevlng  on  overall 

leproveoent  lo  perfot«,oce  In  .  .,at.n  e,..l.  the  cost  of  eddlng  the 

„„.her  of  unit,  required  to  effect  the  Inprovement  .here  tho.e  edded 

/ 

units  operate  at  the  Initial  performance  level. 

pc.^n.«tlng  U$  SD  in  performance  un^.  The  basic 

,rogden-Cronb.ch-Gl.ser  fomul.  «.rk.  In  en,  metric  (.)!  0  end  SD  need 

not  be  enpreaaed  In  dollere.  The  overall  Improvement  In  ey.tem 

performance  is 

Um  -  Ns  SDm  Rxy  Zx 

In  output  unit,  of  performance.  T.  thl.  1.  equal  to  (4) 

nl  fY2  -  Yl)  ■  Ns  SDy  Rxy  ^ 

Substituting  into  (6) 

0$  •  fin  Hs  SPY  Rxy  Zx 

_  Y1 _ 

Formula  (7)  more  conveniently  describes  the  utility  in  dollars 
.election.  Thl.  formula  uae.  SD^.  the  et.ndard  deviation  In  output 
unit,  of  performance,  rather  than  SM.  the  at.nd.rd  deviation  of 

performance  in  dollars • 

...imatlne  SDS  ual"e  t°«t  and  performance  dMa.  Setting  (1)  end 
(7)  equal 

D$  -  Ns  SD$  Ri^  Zx  •  r.n  Wb  SDv  Rxy  Zx 


(7) 


Y1 
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and  solving  for  SD$. 


SD$  •  Cu  SDj 


or,  S2i.q».l.  tfce  co.t  p.t  «»lt  tto..  the  tetlo  of  the  SD  of 
,.rfor«nc.  to  the  loltl.1  l«vol  of  p.rfor«nc..  Thl.  proportion 

h.  ...IX,  ..tlooted  i.  «n,  .Ituntton..  It  1.  l.t.re.tlng  to  not. 
th.t  thl.  p.r.ll.l.  th.  Hunter  h  Schniat  (1982)  notion  thet  SDi  he 

llricea  to  .0..  pernenteg.  of  Hot..  SS.  “  ‘"o  "* 

in  th.  .,.t.n  -  .9019.000.  oopoort.  nnd  p.r.onn.1  -  r.ther  then  ..Ury. 

Booeoer.  ..tl«t..  fro.  both  (7)  nnd  (8)  .r.  .pproprl.t.  only  .hen  th. 
Oerfor^nc.  of  th.  -nit.  In  th.  .,.t«.  1.  to  .  «Jor  .xtent  .  function 
.01.1,  of  th.  perfomanc.  of  th.  Individual  In  th.  Joh  under  Investigation. 
TO  th.  extent  th.t  It  1.  not.  correction,  to  th...  lor».l..  would  he 

required . 

Instrument 

*  gue.tlonnalr.  developed  to  ohfln  ..tlnete.  of  th.  dollar 
ualu.  of  average  (50th  percentile)  and  auperlor  (85th  percentile)  t.nh 
conswnder  perfomance.  and  the  eatlnatad  nunher  of  tanh.  with  auperlor 
ranh  con^nder.  needed  to  equal  th.  p.rfor«nc.  of  a  .tand.rd  conpan,  of 
17  tanh.  with  average  tank  co«»nd.r..  Additional  lt«..  were  need  to 
detemlne  the  confidence  of  respondent.  In  their  .newer..  Dollar  value 
„d  nunher-of-tank  Itws.  were  ,lll-ln-th.-hl.nk  while  confidence 
eattaate.  were  -.Itlple  choice  (within  81.000.  within  810.000. 
within  8100,000.  and  other;  or  exact,  within  1  tank,  wit 
tadta,  within  4-5  tanka,  or  wore  than  5  tank.). 
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Respondents 

The  questionnaire  was  administered  to  A8  Ml  tank  commanders 
enrolled  in  advanced  training  at  a  Continental  U.S.  Army  post  and  to 
five  other  enrollees  (one  M48  Armor  crewman  and  four  U.S.  Marine  tank 
commanders).  All  were  male;  their  racial/ethnic  identification  of  £s 
was  not  obtained.  Their  median  number  of  years  experience  as  a  tank 
crew  member  was  nine.  It  can  be  assumed  that  the  sample  was 
representative  of  the  corps  of  tank  commanders  in  the  continental  U.S. 
Other  data 

Both  the  Superior  Equivalents  Technique  and  the  System 
Effectiveness  Technique  required  information  from  sources  other  than  the 
questionnaire.  To  obtain  another  value  of  average  performance  for  the 
Superior  Equivalents  Technique,  we  used  published  pay  and  allowance 
tables.  In  1983  the  base  pay  for  Army  enlisted  personnel  with  ten  years 
of  service  at  the  ranks  expected  for  tank  commanders  ranged  from  $15,500 
to  $17,000.  Non-taxable  allowances  for  such  items  as  housing  could 
amount  to  more  than  $8,000  for  the  typical  married  tank  commander  with 
dependents.  An  estimate  of  an  equivalent  civilian  salary  would  be  about 
$30,000  per  year. 

Data  for  the  System  Effectiveness  Technique  were  obtained  from 
technical  reports  of  previous  research.  The  cost  of  operating  a  tank 
per  year,  Cu,  was  estimated  by  Wallace  (Note  2)  as  $3,000,000.  This 
amount  appears  not  to  Include  the  20+  year  life  span  expected  for  an 
Army  tank;  we  preferred  the  more  conservative  figure  of  $300,000  per 
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„.t.  erlt.tion  «li«ty  «...tch  o»  t.*  cr«. 

E.to«.  t  Chrl.tUn.eo.  1961)  .ugg«t.B  th.t 

«!«..  for  tb.  „tt.  SBrZll.x.n*.  fr-  .2  to  .5.  We  .elected  the  «re 

conservative  value  of  .2. 

To  permit  compotetlom  o»  21  porpoeee  of  exemple.  It  wee 
oeceetery  to  Identify  .  .election  ratio  and  a  aelectlon  procedure 
yalldlty.  inepectlon  of  tank  doctrine  Indlceted  tank  co«na»ler.  could 
be  choeen  from  tank  driver.,  gunner.,  or  loader..  However.  In  practice 
only  more  eenlor  crew  member,  are  considered.  He  choee  .5  a.  mo.t 
likely  to  reflect  actual  .election  ratios.  Although  higher  validities 
are  often  observed  (Eaton.  Hot.  1  and  1980)  a  conservative  value  of 
BTV  ■  ,3  was  chosen. 

~  Data  fro.  the  questionnaire  and  the  -other  data'  were  ...embled  to 
provide  the  basic  Input  to  each  of  the  three  techniques.  Then,  lor  each 
technique.  SDi»i.  computed,  and  U$  was  determined  on  an  Individual 
tank/tank  commander  besl.,  a.  well  a.  lor  a  eystm.  having  2500  tank,  and 

Cahk  commanders. 

Results 

fiD6  Estimation  Technique 

The  eetlmate.  of  value  for  both  av.rege  and  .uperlor  tank  commander. 
(TO.)  were  slewed  and  very  broad.  For  the  average  TO.  the  medUn 
astlmat.  we.  $30,000.  with  the  first  quertll.  at  $17,000  and  the  third 
quartll.  at  $100,000.  The  distribution  appeered  .om«.h.t  blmodal,-  with 
nineteen  Judga.  giving  eetimate.  betwen  $10,001  and  $30,000.  «>d 
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•hovn  in  Figure  1. 


Insert  Figure  1  about  here 


for  the  superior  TC.  the  median  estimate  was  $50,000  with 

,..„ue  «  .30.000  ..a  ..e  .3.0  «  .300. 000.  ..  a...... 

vas  also  highly  skewed  and  almost  trlmodal.  There  were  ni 
„  n.OOO.OOO  or  -ore.  iBcludln,  on.  .t  positive  inlinit,  .Uh  «  set 
to  .100.000.000.  .0  aisttiOotioo  IS  SW  io  «...  2.  T.. 

,3.S,M0.  ooa  .t.oa..a  aevUtlo.  l.  .U.on.s...  Host  C302)  oi  to. 

..cutat.  -itOln  .10.000.  «itO  1«  not  ...poodio,.  .od  1«  iodictiop 
coofldenc.  Holt,  pteat..  .an  .10.000. 


Insert  Figure  2  about  here 


,.rfomanc.  value  p.ovldad  an  ..timat.  of  §21  ’  ““"'I" 

,.t«ao  avetas.  and  aupa.iot  p..io.aoc.  value  ..ti.t..  .iv.o  0,  eat 

taapoodeot  «.  alao  cooputad.  The  «dlao  of  the  dlffe.encet  ». 

being  $7,000  and  the  third,  $85,00 
15,000,  with  the  first  quartlle  being  $7,u 

of  5  Zx  -  -8  (Hunter  and  Schmidt,  198  ). 
For  a  selection  ratio  of  .5,  zx 
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Lcrpofti.^  r  -  .3,  2x  -  .6.  .  S^of  320,000  1«.  (1)  ,l.ld.a  Hi  ' 

«  t.*  «2.«.d,  Hi  -  $12,000,000  It  2500  t..* 

c»i-nd.i.  vet.  .elected.  »irt  the  SOi  e.tl«t.  of  $15,000.  the  Hi 
eeloee  ete  $3500  end  $9,000,000  respectively.  Values  for  SOi  end  Hi  «' 

shown  in  Table  1* 


Insert  Table  1  about  here 


Superior  Equivalents  Technlqt^ 

The  nodel  response  given  for  the  numher  of  superior  TC.  Judged 
agulvelent  to  17  ev.r.g.  TCs  ...  10.  The  «dl.n  estUat.  .as  9  and  the 
«dlan  confidence  ...  that  the  ..tl«t.  «.  In  error  h,  no  «r.  than  1. 
Ih.  range  of  «  to  12  Included  ahout  952  of  the  rasponses.  About  $52^ 
..Id  their  estls-t.  «.  lu  error  by  no  «.re  than  3.  The  response  -lo¬ 
ws.  Judged  to  he  a  representative  value  of  central  tendency.  The 
distribution  is  shown  in  Figure  3. 


Insert  Figure  3  about  here 


The  estlmatad  value  of  an  average  tank  co.«»nder  ...  $30,000  fro. 
the  SHI  Estimation  Technique,  end  also  fro.  evaluation  of  tank 
oo^andef.  pay  and  .llovanc...  Given  10  auperlor  TC.  Judged  equivalent 
«  17  average  TCs.  and  an  average  TC  -»rth-  $30,000  per  year,  then  a 
auperlor  TC  «uld  he  -«rth-  17/10  tl...  $30,000.  or  $51,000.  This 
figure  1.  .l«.t  the  ...e  ..  the  «dl.n  estimate  of  the  value  of  e 
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t.A  colander  obtained  «lth  th.  21  Eatl-.lon  T.chnl,«.. 

Tnloea  for  Sl«”  ‘  ””  *“* 

noldln*  «!«.  of  S5040  ond  $12,600,000  toapoctloaly.  Ih...  «lne.  ore 
alto  shown  in  Table  1* 

System  Ef feetlveneaa  Technique 

Th.  ».1«.  of  21«“  ““ 

oalne.  Cu  -  $300,000  ond  21^.-  -2.  21  ’  I*'’-'’'"- 

^  for  I  and  ?500  tank  eonnand.t  ..l.ctlon.  «r.  eo.„.uted  na  $14,400  and 

$36,000,000  rnapectlely •  Theae  ore  ahown  In  Table 

Discussion 

The  varlou.  «thoda  of  eatlnatlng  and  2  O-  "«  “> 

teaulta.  dUhoogh  rater,  providing  e.ttaatea  lor  the  $2 
..tl«tlon  technlgue  Indicated  confidence  In  their  indgnenta.  the 
dUtrlbotlon  of  reeponae.  1.  far  fro.  confortlng.  The  ertrene  akevlng 
of  lodg^nt.  and  their  bl/trl-  «dal  natnre  auggeated  that  all  reaponae. 
«ve  not  haaed  on  the  ...e  conalderatlona  and  value..  Thl.  «.pport.  our 
concern  about  the  difficulty  of  «klng  auch  ludgnent.  .hen  the  coat  of 
contracting  o-t  the  «rk  1.  unkno«>  and/or  other  financially  Intangible 
lector,  are  Involved.  Such  1.  freguently  the  caae  for  public  enployee. 
auch  aa  th.  nlUtary  .hen  private  Indu.try  counterpart,  are  nonerlatent. 

son.  conflra^tory  evidence  for  th.  21 
fro.  the  almllarlty  of  teaulta  obtained  ualng  the  Superior  Equlvale 
lechnlgue.  Th.  value  of  average  perfon»nce.  $30,000,  .a.  th.  a... 
regardlea.  of  .h.ther  obtained  fro.  th.  Inatrunent  (need  In  th.  21 
Eatlnatlon  T.chnlgu.)  or  fro.  the  .Independent*  pay  and  all«,anc.. 
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tod  the  r.tl.  .f  10  »*.  .^th  .uptol.r  c.-«d.r. 

17  »lth  «.r.g.  c<»tond«.  producd  .  SPi  •* 

,„ctlc.U,  to».l  t.  to.  010.000  ligur.  to.  SHi  *«*“'*- 

Technique. 

,0th  t.chnl,«..  yUltod  .ImlUt  r..uU.,  ..d  both  .r.  within 
„ng.  of  *0-701  of  .nnu.l  ..Uty  ..  nntod  by  tont.r  .nd  Sctoldt  (1982) 
nod  (1983)  to  b.  typitol.  ««  toll....  bo«v.r.  th.t  to.  Supotlor 
Equivalents  Technique  is  the  method  of  choice  in  situations  where 

.npotvloot.  .»  tote  ..on.to«^ 

or  in  t.Utlv.  output  of  Indl.ldu.ls.  then  In  dollar  terns.  In 
.nob  .Ituetlons,  .uperuLof.  ludgnent.  of  r.letlue  perfomenoe  .een 
credible  then  their  estlnete.  of  toe  doller  velue  of  .«r.,e  .nd 
.nperlor  perfomenc.  The  reUtl.e  verl.nces  of  toe  dl.trlb.tlon.  of 
the..  ..lues  in  thl.  research  would  .een  to  .upport  this  point 

,..plt.  tholr  conslstoncy.  we  wonder  whether  both  8^  .«toetlon 
.nd  .uperlor  .,ul..l.nt.  tothod.  produce  underestltotes.  P.rhep. 
respondent,  estlnete.  .re  psy/ellowence-besed.  nnd  these  el«.ent.  nre 
not  indexed  to  toe  ttoh  connander  Job  In  terns  of  'reel  worth., 
aespondenf.  ..tlnete.  of  toe  welu.  of  nver.ge  perfomsnc.  were  obt. 
..p.r.t.ly  fron  our  ee.luntlon  of  pay  aod  allowauce..  but.  of  course^^^ 

t,nk  connander.  know  well  their  renuneratlon.  Man,  respondent,  nay 

f  ...  avetane  tank  connander  to  be  worth  what  he 
Judged  the  performance  of  an  averag 

1.9  pftldm 

Higher  values  f or  Shd  «-  »! 

These  are  probably  as  accurate  as  any  estimates.  They  were  ba 
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SDy/Yl  ratios  from  actual  field  data  and  cost  estimates  which,  while 
crude,  may  more  accurately  reflect  reality.  It  is  also  plausible  that 
these  Systems  Cost  Estimates  are  below  the  “true-  values.  A  tank 
probably  costs  more  than  $300,000  per  year  to  operate,  if  not  the 
$3,000,000  proposed  by  Wallace  (Note  2);  and  .2  is  the  lower  bound  for 

the  ratio  of  SDy  to  Yl. 

One  could  argue  that  improved  performance  of  taidc  commanders,  the 
basis  for  the  SD$  Estimation  and  Superior  Equivalents  Techniques,  has 
only  a  partial  impact  on  improving  tank  performance,  the  basis  of  the 
system  effectiveness  procedures.  An  empirical  question  is  the  size  of 
the  contribution  of  the  tank  commander  to  the  crew  and  tank;  we  do  not 
yet  know  the  answer.  However,  both  analytical  and  rational  Judgments, 
as  well  as  the  lore  within  the  armor  community,  suggest  that  the 
performance  of  the  tank  is  largely  a  function  of  the  performance  of  the 
tank  commander.  It  was  the  assumption  of  the  overwhelming  impact  of  the 
tank  commander  on  tank  performance  that  led  to  our  initial  thoughts  on 
the  Systems  Effectiveness  formulae  derivations. 

The  two  additional  limitations  we  see  of  the  Systems  Cost  Technique 
are,  first,  the  problem  of  estimating  the  cost  of  a  unit  in  the  system 
and,  second,  its  restriction  to  those  situations  in  which  the  quality  of 
performance  can  be  indexed  by  quantity.  Wallace’s  (Note  2)  estimate  of 
unit  cost  per  tank  (^)  may  seem  excessive.  Nevertheless,  such  estimates 
can  be,  and  frequently  are,  made  in  accounting  departments.  It 
also  can  be  adjusted  if  it  appears  out  of  line.  Second,  quality  as 
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..t  b.  ^.nlndful  in  •»-  .ltu.tl.«. 
,,«p.ctlve  ...r.  .bould  ,»..tlo».  b,»ev«.  Whether  ,ueU..tl« 
indlee.  «y  not  be  «de  into  ,».ntlt.tl.e  one.,  m  nxem-U.  • 
police  deyer»ent  decide  thet  conelctlo.  .1  one  -.rderer  1. 

.,„iv.lent  to  the  conviction  oi  eeverel  bnrdlere.  U  1.  Uhely 
r..t  ench  .,-tlon.  nre  beln*  need  to  conp.te  periomence  oi 
different  individoele.  nlbeit  InfomeUy  nnd.  poeeibly. 


inconsistently* 
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Footnote 


A  pr.llAlo.tJ  «r.lon  of  thl.  t...arch  w.  pteseotod  .t  the 
.....otloo  of  th.  Aoerlcao  fjchologlcl  A..ocU.loo.  lo.  Aogol.., 


Cllforol..  in  Autost  1983.  l.«r.oc.  M. 
provided  many  helpful  suggestions. 


Hanser  and  Donald  F.  Haggard 
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Table  1 

of  SD$  and  Examples  of  UtlllSS 

SD$  U$  or  utility 
per  tank 
(Ns  »  1) 


U$  or  utility 
per  system 
(Ns  »  2500) 


SD$  Estimation  Technique 

Difference  between  medians 

$20,000 

$  4,800 

Median  of  differences 

$15,000 

$  3,600 

Superior  Equivalents  Technique 

$21,000 

$  5.040 

System  Effectiveness  Technique 

$60,000 

$14,400 

$12,000,000 
$  9,000,000 
$12,600,000 
$36,000,000 
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Figure 


Plgure  Captions 
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rigor,  t.  E.,tl«tl..  of  "oil*" 
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rigor.  3.  E.tl-tloo  of  n«ob.r  of  t.ok.  with  .operlor  t.nh 
cootwnd.r.  .,o.lt«E  »  ‘.oh.  with  .v.r.g.  t.oh  ..-..na.r.. 
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I.  GENERAL  INSTRUCTIONS 


.  Testing  Method 

The  Officer  j®”y^®^adminlstered  in^accordance  with  the 

,nd  OSB.4,  DA  ),  Jorth  in  Chapter  III,  "Test 

)rinciples  of  test  ^  Personnel  Tests.  In  addition, 

Ldministration,"  in  this  manual  for  the  OSB  are  to  be 

^he  specific  directions  given  i  .  ^  changes  in  the 

rollowed  without  deviation.  No  Each  testing  session 

wording  of  these  directions  are^pe^  administrator,  and  for 

is  to  be  conducted  ^  trained  proctor  for  each  additional  25 

groups  greater  than  25,  one  trainee  p 

examinees. 

Each  form  of  the  j^|^d°Part  Il!^^ELh^examinee  is  to 

separate  test  booklets,  p  ^  jj  same  form.  The  test 


Order  of  Administraticni 

Part  I  should  be  d‘*“''j®J?'"®|.ugn|e’aUovis'*extra  time  for 
:i:;erll‘fntroJurJy"r»a?hs  that  do  aot  have  to  be  repeated  for 
>he  second  part. 


!.  Time  Required 

About  50  minutes  required  for^adminlstering^eaoh^part  o 

^he  Officer  Selection  Battery.  reading  directions,  and 

required  for  distributing  timed,  but  examinees  should 

reTr^e"d^^frrara  ready%ace  so  they  will  finish  during  a 

class  period. 

T.e  two  test  «^i\rtrci^mpretflf  Sermfss^bl:  to^ 

separate  class  periods,  8  j^gtely  after  the  other  in  a 

administer  the  two  J”®  s  done,  have  the  examinees  stand 

s^re^t^rbU-rerihe  end  of  Part  I  and  the 

beginning  of  Part  II. 
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4. 


Materials  Bequired  for  Each  Test  Period 
For  each  examinee 


1)  One  test  booklet 

2)  One  answer  sheet  -g^cils  with  erasers 

II  irrafcrp^lpfr  fo^e^Seet  should  he  sufficient) 

B.  For  the  examiner 


1) 

2) 

3) 

4) 


5) 

6) 


A  copy  of  this  manual 
A  test  booklet 

Chalk^and  chalkboard  or  large  paper 
fo?  deroLtratlng  answer  .narking, 
and  for  posting  time  remaining 

mil  limit  extra  scratch  paper 


5.  Test  Security 

i.  .<11  viavA  the  responsibility  for 

The  test  sheets  during  the  examination. 

^^^rthLfprrlo^nfwL^wJir^e  involved  in  test^ 

S^r^^srrelert  uSdl?  Tfcrlfd^indfr-control  of  a  designated 
Test  Control  Officer. 


6.  Advance  Planning 

Advance  planning  J .|^;^®^uS%eforrthe''t«rperiodr‘in 

“rer*?o^?:proye-thrq^ai;iy:r?fst%d.lnlstratlon. 

g  Be  in  the  room  when  «•►>?, examinees  errl«-  "^"a^are 

exLtres"b;‘sLwng°?iem  Slth  empty  seats  between  them, 
if  possible. 

B.  Decide  upon  '‘“g?iactlng'*test5  and'answer  sLets  In 

t;;f^irii;irr5oririfg;5e‘tfe  tests. 

C.  Plan  hO"  b“anlSI?ed  br'ereading^thrtlsf  dl?ectlons 

tions  should  be  answere  y  reread  them.  Do  nai- 

a  response  to 
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further  Questioning  about  directions  might  be  "Do  the 
besl  you^canr?  aAot  permitted  to  tell  you  any  more 
than  what  the  printed  directions  say. 

D.  Plan  to  post  the  "Minutes  Remaining"  every  10  minutes  on 
the  chalkboard. 

E  Motivating  the  examinees  Is  very  Important.  Some  remarks 
hive  bUn  prepared  to  be  read  to  them,  but  if  you  also 
lllLairb?  ylur  attitude  that  jou  believe  the  test  is 
important,  it  will  help  even  more. 


II.  ADMINISTERING  THE  TESTS 


1 .  Distribution  of  Testing  Materials 


Immediately  prior  to  the  testing  session,  place  one  set  of 

lirilleriniJl4ol"rareJ:Mrtl:%Mltfs1n  Talh^rxSnee.s  ’ 

desk,  face  down. 

2.  Introductory  Remarks  by  Test  Administrator 

Each  testing  session  will  be  introduced  by  f 

siruirii%:?MrariiSU?risi‘raigii:r^aid=iieirrairages  must 

BE  READ  EXACTLY  AS  WRITTEN. 

You  will  begin  each  test  session  by  saying. 

You  are  about  to  take  a  test  '^sed  by  the  Army  to 
help  select  officer  candidates  and  Advanced  ROTC 
cadets.  You  will  complete  two  test  booklets, 
either  both  today  or  during  periods. 

These  booklets  will  cover  a  number  of  different 
subject  areas. 

One  of  the  things  the  Army  wants  to  know  is  what 
the  best  talents  and  skills  of  officer  candi¬ 
dates  are,  so  the  tests  cover  many  topics. 

Some  questions  will  be  easy  and  some  will  be 
hard.  Don't  be  discouraged  by  the  hard  ones. 

Just  work  as  quickly  and  accurately  as  you  can. 
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Slank’.  «^"*^h°en  print  your  middle  Initial. 

rwiu“l"t!”^srln"ira^»umtrr%  pencil  llKe 
the  one  you  have  been  given. 


Pause  while  examinees  do  this.  Then  say: 


.  j  e..:.rah  ipttpr  of  vour  nsme  by  making  a 

name.  If  you  have  any  questions,  rais  y 
hand . 


Pause  While  examinees  do  this.  Then  say: 


NOW  look  at  the  lower  le«  ^jner  of  the  oover^^ 

of  vour  test  booklet.  There  is  -5 - 

pM^ed  CT?Fe.  Co'55  this  number  in  Block  2 
under  DA  Form  on  your  answer  sheet. 


Pause  while  examinees  do  this.  Then  say: 


I  Block  3  on  your  answer  sheet,  headed  SSN,  is  for 
tjiocK  3  j  ^pcuritv  Nurnb6r«  Enter 

‘-iS  s' 

the  nine  digits  of  g  heavy  mark 

digit,  including  zeroes,  by  ® 

in  the  appropriate  numbered  space  m  each 

column.  Do  this  now. 

pause  While  examinees  fill  in  this  information.  Then  say: 

St“re°c?ronrrn^tre%nsfd^/of^"tL^??onr:ovar 

while  I  read  them  aloud. 

Read  the  directions  printed  inside  the  front  cover  of  the 
booklet.  Then  say: 

I  Do  you  have  any  questions  about  what  you  are  to 
do? 


test 
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rtelay  the  start 

t  the  examinees  oeiay 

answer  any  |?relevant‘'°queftlons .  Tnen  say: 

0^  testing  «tn  arne  S 

,  If  you  ise  your  hand.  J°Vj.”you  work  1 

the  test,  raise  y  gtions,  on  any  1 

ra^U  -/^.onn  spen^ 

ro%r/arrs  .r .i--r%raV.  -  1 

doubt, 

1  Page  1  begin. 

3.  Moni^orJJiSJ^  ^around  the  ^^CheS’^^ 

ds  examlneas  fagin  v.orkin|j_  w|_.^  ^  ^^ler^af  b«n  blaokenad 

that  fhe^  pnopat 

aaPaaiallV^/f^fai  security  number  „„3inlng  on 

and  that  the  so  “i’!  The  purpose  m 

At  3^°'^^d*°e”g'^'!'^^”3o’^roi’'‘'^^®%o®aid%xamine  in  ^f^iplcter 

^rtunf^r^  exceptionally 

,...,;s  “iSi  Ss‘"“‘"-'‘'””  ■“  “ 

option  “f  pf  JfavUed  from  creating 

rtt^u”wfrklng.  tbelr  answer  sheets 
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III.  SCORING  THE  OSB 


1.  Hand  Scoring 

Scan  the  answer  sheet.  If  more  than  one  answer  to  any 
question  has  been  marked,  draw  a  red  line  through  answer 
spaces  to  that  question. 

Place  the  scoring  stencil  that  corresponds  to  the  OSB  Form 
number  and  Part  number  of  the  answer  sheet  on  top  of  the  answer 
sheet.  Make  certain  that  the  stencil  is  properly  aligned. 

Count  the  number  of  black  marks  that  show  through  the  holes 
in  the  stencil.  Do  not  count  any  marks  that  have  a  red  line 
through  them  (remember,  these  are  questions  to  which  the  examinee 
marked  more  than  one  answer,  so  none  of  them  is  counted  as  cor¬ 
rect)  . 

The  total  score— total  number  of  right  answers— should  be 
written  on  the  answer  sheet  in  the  place  provided  by  the  cut-out 
on  the  stencil.  The  scorer  or  another  person  should  recount  the 
black  marks  to  make  sure  the  total  is  correct. 


2.  Converting  to  Army  Standard  Scores 


Use  the  appropriate  conversion  table  to  convert  the 
examinee’s  raw  score  (total  right)  to  an  equivalent  Army  Standard 
Score.  Be  certain  that  you  are  using  the  proper  table;  it  is  the 
same  color  as  the  test  booklets  for  the  form  used,  and  has  the  form 
number  on  it.  If  you  have  tested  high  school  students,  there  is 
a  separate  conversion  table  for  use  with  their  scores. 


Find  the  Raw  Score  in  the  left  column,  and  read  the  cor¬ 
responding  Army  Standard  Score  in  the  adjacent  column.  Record  the 
Army  Standard  Score  on  the  answer  sheet,  next  to  the  raw  score 

The  Army  Standard  Score  becomes  the  official  Officer  Selec¬ 
tion  Battery  score,  which  is  recorded  in  any  official  records  or 
notices  required  by  the  Department  of  the  Army,  Department  of 
Military  Science  or  ROTC. 


3.  Machine  Scoring 
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EXKCUTIVE  SUMMARY 


This  paper  describes  the  potential  benefit  of  using  information  from 
Project  A  teste  in  selecting  soldiers  for  the  Excellence  in  Armor  Program 
(EIA).  To  evaluate  the  Impact  of  the  new  tests,  we  compared  end-of- 
training  (EOT)  measures  for  EIA  soldiers  in  the  Project  A  database  with 
those  of  non-EIA  soldiers  (19E  and  19K)  and  with  samples  formed  from  the 
same  pool  by  applying  two  test-based  decision  rules.  Finally,  we  examined 
the  effects  on  EOT  measures  of  applying  the  decision  rules  to  the  EIA 
soldiers. 

Two  different  test-based  rules  were  generated.  Each  was  devised  to 
designate  approximately  the  top  30  percent  of  the  pool  of  soldiers 
earmarked  for  19  CMF.  One  strategy  emphasized  the  Assessment  of 
Background  and  Life  Experiences  (ABLE) ,  a  measure  of  leadership  potential 
(ABLE  Emphasis);  another  equally  emphasized  ABLE  and  a  set  of  spatial  and 
psychomotor  tests  (Equal  Emphasis).  One  important  difference  between  the 
existing  and  test-based  strategies  is  the  point  at  which  information 
becomes  available  to  decision  makers.  The  existing  strategy  relies  in 
information  collected  after  soldiers  have  begun  OSUT.  The  new  strategy 
generates  information  before  training  begins. 

The  new  test-based  rules  were  evaluated  in  a  sample  of  2,372  soldiers 
in  MOS  19E  and  19R.  All  soldiers  had  graduated  from  One  Station  Unit 
Training  (OSUT),  162  of  these  in  the  EIA  program.  In  addition,  these 
soldiers  had  been  tested  on  Project  A  predictors  as  receptees  and  had 
completed  EOT  measures.  The  EOT  measures  Included  peer  and  supervisory 
ratings  and  scores  on  an  MOS-speciflc  EOT  knowledge  test.  We  first 
applied  the  new  rules  to  the  total  sample  of  2,372  soldiers  and  then  the 
smaller  subsample  of  162  EIA  graduates.  Our  purpose  was  to  compare  the 
two  new  strategies  with  each  other  and  with  the  existing  strategy. 

We  found  that  the  existing  strategy  for  tracking  trainees  is  working 
well.  It  produces  EIA  graduates  who  are  seen  as  superior  to  Normal  Track 
(NT)  soldiers  by  their  superiors  and  peers  and  who  outscore  their  NT 
counterparts  on  EOT  tests. 

The  soldiers  designated  by  the  new  strategies  scored  similarly  to  the 
EIA  graduates  on  the  EOT  tests  and  about  halfway  between  the  ET  and  NT 
sample  on  the  peer  and  supervisory  ratings.  The  Equal  Emphasis  rule 
proved  superior  to  ABLE  Emphasis  rule  due,  in  part,  to  the  addition  of  the 
predictive  ability  of  the  spatial  and  psychomotor  tests.  In  the  full 
sample,  both  decision  rules  produced  similar  peer  and  supervisory  ratings 
but  the  Equal  Emphasis  rule  produced  better  performance  on  the  EOT  tests. 
In  addition,  the  Equal  Emphasis  rule  created  a  larger  pool  of  soldiers 
than  did  the  ABLE  Emphasis  rule. 

We  also  applied  the  new  test-based  strategies  to  the  EIA  sample  to 
determine  how  closely  they  replicated  existing  selection  procedures.  The 
new  test-based  strategies  selected  only  40-50  percent  of  those  selected  by 
existing  strategies.  However,  those  EIA  graduates  who  were  selected  by 
the  test-based  strategies  were  superior  to  the  remaining  soldiers  in  all 


3 


respects.  Thus,  the  new  tests  selected  the  "cream  of  the  crop"  within 
EIA. 

Based  on  our  evaluation,  we  conclude  that  the  existing  Information 
sources  and  new  test-based  information  are  complementary  and  appear  to 
work  well  together  to  select  the  best  EIA  candidates.  Ve  also  found  that 
a  test-based  strategy  giving  equal  weight  to  the  ABLE  and  a  composite  of 
the  spatial  and  psychomotor  tests  was  superior  to  one  emphasizing  the  ABLE 
alone.  As  in  past  research,  we  expect  that  the  spatial /psychomotor 
component  of  the  Equal  Emphasis  strategy  will  yield  improvement  in  the 
"can  do"  hands-on  gunner  performance  (such  as  that  measured  by  the  UCOFT), 
and  that  the  ABLE  component  will  predict  the  "wlll-do"  components  of 
performance. 


4 


Introduction 


The  purpose  of  this  paper  is  to  provide  information  to  the  U.S.  Army 
Armor  School  to  support  their  planning  for  the  Gunner  Selection  and 
Sustainment  Program.  The  Excellence  in  Armor  Program  (EIA)  calls  for  the 
identification  of  outstanding  soldiers  in  the  Armor  and  Cavalry  field.  It 
was  created  to  provide  incentives  to  encourage  these  soldiers’  continued 
superior  performance  and  improve  their  retention  rate.  Soldiers  are 
identified  for  the  Excellence  Track  (ET)  in  their  seventh  week  of  One 
Station  Unit  Training  (OSUT)  for  early  advancement  and  specialized 
developmental  experiences.  This  paper  compares  attributes  and  performance 
of  soldiers  who  are  in  the  Excellence  in  Armor  Program  to  those  in  the 
Normal  Track  (NT).  In  addition,  it  evaluates  the  effect  of  two  potential 
rules  for  making  decisions  based  on  the  Project  A  tests. 


Method 


Subjects 


Subjects  were  2,372  soldiers  in  the  19  CMF  (578  in  MOS  19E  and  1,794  in 
MOS  19K)  who  entered  the  Reception  Battalion  at  Ft.  Knox  during  the  period 
from  August  1986  to  November  1987  and  subsequently  completed  the  full 
Project  A  predictor  battery.  Of  these  soldiers,  162  have  since  completed 
OSUT  in  the  Excellence  Track  and  2,210  have  completed  the  Normal  Track. 


Measures  of  Soldiers’  Attributes 

Cognitive  Ability.  Before  entering  military  service,  all  soldiers 
completed  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB).  The 
Armed  Forces  Qualification  Test  (AFQT) ,  a  composite  of  four  of  the  ten 
ASVAB  subtests,  served  as  the  measure  of  cognitive  ability. 

Psvchomotor  Skills.  Two  computerized  psychomotor  tests  were  included 
from  the  Project  A  Battery:  One-handed  Tracking  and  Two-handed  Tracking. 
The  former  measures  steadiness  and  precision;  the  latter  measures 
coordination  and  dexterity. 

Spatial  Ability.  Two  of  the  Project  A  paper-and-pencil  tests  of 
spatial  ability  were  included.  The  Maze  Test  assesses  the  ability  to  scan 
a  field.  The  Orientation  Test  measures  the  ability  to  maintain  one’s 
perspective  or  bearing  with  respect  to  some  object  when  it  and  its 
component  parts  have  been  rotated. 

Spatial  and  psychomotor  test  scores  were  formed  into  a  single  composite 
score. 

Leadership  Potential.  Leadership  Potential  was  assessed  using  four 
scales  from  the  Assessment  of  Backgroimd  and  Life  Experiences  (ABLE),  one 
of  the  noncognitive  tests  included  in  the  Project  A  predictor  battery. 
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The  four  ABLE  scales  were:  Dependability,  Adjustment,  Work  Orientation 
and  Social  Dominance.  A  composite  was  formed  by  summing  the  scores 
obtained  on  the  68  items  Included  in  these  scales. 

Performance  Measures 


As  part  of  ongoing  Project  A  data  collection,  soldiers'  performance 
was  assessed  at  the  end  of  OSUT.  This  subset  of  Project  A  criterion 
measures  was  used  in  the  present  analyses.  These  measures  are  described 
below. 

Ratings .  Supervisors  and  peers  rated  soldiers  on  a  series  of 
behaviorally  anchored  rating  scales.  Ratings  were  made  on  a  seven-point 
scale.  Dimensions  included:  Technical  Skill,  Effort,  Following 
Regulations  and  Orders,  Military  Appearance,  Physical  Fitness,  Self- 
Control,  and  Leadership  Potential. 

End-of -Training  (EOT)  Knowledge  Tests.  At  the  end  of  training, 
soldiers  completed  an  MOS-speclfic  knowledge  test  addressing  the  material 
covered  in  training. 


Development  of  Decision  Rules 

New  test-based  decision  rules  were  developed  for  evaluating  the 
relation  of  scores  on  the  Project  A  tests  to  selection  into  EIA  and  to 
measures  of  performance.  In  developing  these  rules,  we  considered  the  two 
kinds  of  abilities  assessed  by  the  Project  A  predictors:  leadership 
potential  (as  measured  by  the  ABLE)  and  spatial /psychomotor  abilities  (as 
measured  by  the  two  paper-and-pencil  tests  of  spatial  ability  and  two 
computerized  tests  of  psychomotor  skills).  Ve  formed  our  decision  rules 
by  examining  flow  rates  into  the  19  CMF.  Norm  tables  for  these  decision 
rules  were  based  on  the  scores  of  over  20,000  soldiers  in  combat  MOS  (IIB, 
13B,  16S,  19E  and  19K)  who  took  these  tests  in  Project  A  during  the  period 
from  August  1986  to  November  1987. 

Two  different  strategies  were  Investigated,  each  with  the  goal  of 
designating  the  top  30  percent  of  receptees.  We  selected  30  percent  as  a 
target  because  doing  so  would  create  a  large  enough  pool  to  accommodate 
practical  concerns  such  as  the  variable  flow  of  soldiers  into  the 
Reception  Battalion,  and  the  need  to  Identify  a  large  enough  pool  of 
highly  qualified  soldiers  to  permit  selection  of  top  quality  soldiers  for 
other  programs  as  well.  The  two  strategies  differed  in  the  amount  of 
influence  afforded  to  each  of  the  two  kinds  of  measures. 

One  decision  strategy,  (Equal  Emphasis)  gave  equal  weight  to  the  ABLE 
and  the  spatial /psychomotor  composite.  To  be  included  in  the  pool, 
soldiers  had  to  have  scores  on  both  sets  of  tests  comparable  to  the  top  55 
percent  of  the  20,000  soldiers  in  the  comparison  group  (l.e.,  above  the 
45th  percentile  on  each  kind  of  test).  In  effect,  this  rule  assigns  equal 
importance  to  gunnery  aptitude  and  to  leadership  potential. 
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A  second  strategy  (ABLE  Emphasis)  afforded  more  weight  to  the  ABLE. 

This  strategy  formed  a  pool  of  soldiers  who  scored  well  on  the  ABLE,  while 
screening  out  those  with  poor  spatial  and  psychomotor  skills.  To  be 
Included,  soldiers  had  to  score  better  on  the  ABLE  than  67  percent  of  the 
soldiers  In  the  comparison  group  (l.e.,  above  the  67th  percentile).  In 
addition,  this  decision  rule  excluded  soldiers  who  scored  below  the  10th 
percentile  on  a  composite  formed  from  scores  on  the  spatial  and 
psychomotor  tests.  Thus,  this  rule  stresses  leadership  potential  but 
excludes  examinees  who  have  very  low  aptitude  for  gunnery. 

Analyses 

One  purpose  of  the  analyses  was  to  compare  the  pool  of  soldiers  who 
were  selected  for  EIA  using  existing  strategies  with  pools  resulting  from 
each  of  the  two  decision  rules  described  above.  For  this  comparison, 
subgroups  were  created  by  applying  each  decision  rule  to  the  total  sample 
(N  -•  2,372).  The  Equal  Emphasis  decision  rule  produced  a  pool  of  746 
soldiers;  the  ABLE  Emphasis  decision  rule  produced  a  pool  of  592.  A 
third  subgroup  consisted  of  EIA  graduates  (N  162).  As  a  basis  of 
comparison,  those  soldiers  who  did  not  graduate  from  EIA  are  also  Included 
(N  -  2,210).  Means  on  predictor  and  criterion  variables  were  then 
computed . 

A  second  purpose  of  the  analyses  was  to  compare  the  two  new  decision 
rules  to  the  existing  one.  This  was  done  by  applying  the  decision  rule  to 
the  sample  of  EIA  graduates  to  see  how  many  would  be  Included  by  each 
rule . 


Results 


Table  1  shows  means  on  predictor  and  criterion  variables  for  the  four 
different  subgroups.  The  existing  decision  strategy  and  the  two  test- 
based  strategies  select  soldiers  who  are  clearly  superior  In  all  respects 
to  those  who  were  not  EIA  graduates.  Some  differences  emerged  among  the 
pools  resulting  from  the  three  selection  strategies.  As  would  be  expected, 
the  Equal  Emphasis  decision  rule  produced  the  highest  scores  on  the 
spatial /psychomotor  composite  and  the  ABLE  Emphasis  rule  produced  the 
highest  ABLE  scores.  Compared  with  unselected  NT  soldiers,  OSUT 
graduates  meeting  either  the  Equal  Emphasis  or  ABLE  Emphasis  decision  rule 
scored  consistently  higher  on  the  EOT  ratings  and  tests.  Compared  with 
Equal  Emphasis,  ABLE  Emphasis  produces  slightly  higher  means  on  the 
ratings.  Conversely,  Equal  Emphasis  produces  a  slightly  higher  score  on 
the  Training  Knowledge  Test.  The  existing  selection  procedure  produces 
performance  ratings  that  are  consistently  higher  than  those  produced  by 
the  test-based  strategies;  means  on  the  Training  Knowledge  test  are 
comparable  to  those  produced  by  the  Equal  Emphasis  rule. 

The  next  step  In  the  analyses  was  to  apply  each  of  the  two  new  test- 
based  decision  rules  to  the  EIA  sample  (N  >■  162).  The  two  test-based 
decision  rules  do  not  reproduce  existing  decision  strategies.  The  Equal 
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Emphasis  rule  would  result  in  the  selection  of  77  EIA  graduates  (about  50Z 
of  the  total);  ABLE  Emphasis  would  select  66  (42Z)  of  the  EIA  graduates. 

Table  2  compares  EIA  graduates  who  would  have  been  selected  using  the 
new  decision  rules  to  those  who  would  not  have  been  selected  and  to  the 
full  subsample  of  EIA  graduates.  Both  decision  rules  select  soldiers  who 
tend  to  score  better  than  the  average  EIA  graduate  on  the  predictors  and 
criteria.  In  all  respects,  those  who  would  have  been  selected  using  the 
Equal  Emphasis  rule  are  superior  to  those  who  would  not  have  been 
selected.  The  most  notable  differences  are  In  the  three  predictor  scores, 
the  peer  and  supervisory  ratings  of  technical  skill  and  performance  on  the 
Training  Knowledge  Test.  The  ABLE  Emphasis  rule  produces  similar  but  less 
striking  results. 


Discussion 

The  present  investigation  compared  existing  procedures  for  tracking 
trainees  with  two  new  test-based  decision  strategies.  The  first 
strategy.  Equal  Emphasis  Includes  soldiers  whose  scores  are  comparable  to 
those  In  the  top  55  percent  of  the  comparison  group  on  both  the  ABLE  and  a 
composite  of  spatial  and  psychomotor  tests.  The  second  strategy,  ABLE 
Emphasis,  selects  soldiers  who  are  comparable  to  the  top  third  of  the 
comparison  group  on  ABLE  and  screens  out  those  comparable  to  the  bottom 
tenth  on  the  spatial  and  psychomotor  composite. 

The  data  suggest  that  the  existing  strategy  for  tracking  trainees  Is 
successfully  producing  a  group  of  EIA  graduates  who  are  seen  as  superior 
by  their  supervisors  and  peers  and  who  score  well  on  EOT  tests.  However, 
the  data  also  suggest  that  much  can  be  gained  from  incorporating  the 
Project  A  Tests  Into  existing  strategies. 

The  existing  strategy  and  the  new  test-based  rules  use  different 
sources  of  information  in  designating  soldiers.  Therefore,  It  is  not 
surprising  that  the  existing  and  test-based  strategies  would  produce 
visibly  different  results.  The  new  test-based  rules  rely  solely  on 
information  available  before  OSUT,  whereas  the  existing  strategy  takes 
advantage  of  seven  additional  weeks  of  performance-based  information  from 
training  context. 

In  some  ways,  our  comparison  between  existing  and  test-based  strategies 
is  stacked  in  favor  of  the  existing  strategy.  The  peer  and  supervisory 
ratings  may  have  been  positively  influenced  by  raters*  knowledge  that 
certain  individuals  had  been  targeted  for  EIA.  Further,  ET  soldiers 
received  developmental  experiences  during  training  that  were  not  made 
available  to  soldiers  in  the  NT.  In  addition,  the  EIA  sample  contains 
only  those  who  successfully  completed  the  program.  A  fair  percent  of 
those  who  enter  EIA  during  OSUT  drop  out  before  training  ends.  If  we  had 
been  able  to  obtain  predictor  test  scores  and  EOT  measures  for  all 
soldiers  who  had  been  in  the  EIA  program  (including  dropouts),  their 
scores  on  the  peer  and  supervisory  measures  may  have  been  more  similar  to 
those  of  soldiers  selected  with  one  of  our  new  decision  rules. 
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Our  application  of  the  new  test-based  decision  rules  to  the  162  EIA 
graduates  served.  In  part,  as  a  control  for  the  effects  of  criterion 
contamination.  All  soldiers  In  this  group  had  been  exposed  to  similar  In- 
tralnlng  Influences.  Within  the  group  of  EIA  graduates,  we  compared 
predictor  and  criterion  scores  of  those  who  would  have  been  selected  bj 
each  rule  to  those  who  would  not  have  been  selected.  When  the  new 
decision  rules  are  applied  to  the  EIA  graduate  sample,  the  best  EIA 
graduates  (the  "cream  of  the  crop")  are  selected.  (See  Table  2.) 

The  means  on  Table  2  suggest  that  an  Equal  Emphasis  strategy  appears  to  be 
superior  to  ABLE  Emphasis.  As  an  added  advantage,  Equal  Emphasis  creates 
a  larger  pool  of  eligible  soldiers. 

In  conclusion,  our  Intent  is  not  to  work  against  the  present  selection 
system,  but  to  point  out  that  the  test  scores,  when  used  in  conjunction 
with  existing  procedures,  can  produce  an  even  better  pool  of  soldiers  for 
the  Excellence  Track.  We  expect  that  a  selection  process  which  emphasizes 
spatial  and  psychomotor  abilities  equally  with  leadership  potential,  like 
the  Equal  Emphasis  rule,  will  provide  additional  advantages  later  In 
soldiers'  careers.  Past  research  (Smith  &  Graham,  1987)  suggests  that 
this  strategy  will  yield  Improvement  In  the  "can  do"  hands-on  gunner 
performance,  such  as  that  measured  by  the  UCOFT,  because  of  the  additional 
emphasis  It  gives  to  the  spatial  and  psychomotor  tests.  Furthermore,  the 
ABLE  predicts  the  "will-do*  components  of  performance  (Hough,  Cast,  White 
Se  McCloy,  1986).  Thus,  we  can  expect  soldiers  who  have  higher  ABLE  scores 
to  be  more  willing  to  perform  In  training  and  on  the  job. 
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Table  1:  Comparison  of  Means  on  Predictors  and  Criteria:  New  Decision 
Rules  vs.  Existing  Strategy 

i 


Variable 


Equal  ABLE 

Non-EIA  Emphasis  Emphasis  EIA 

N  -  2,210  N  -  753  N  -  592  N  -  162 


Predictors 


AFQT 

59 

67 

65 

64 

ABLE® 

44 

74 

86 

57 

Spatial /Psychomotor^ 

51 

78 

65 

65 

Criteria 


Peer  Ratings 


Technical  Skill 

4.35 

4.76 

4.72 

5.23 

Effort 

4.03 

4.28 

4.27 

4.41 

Following  Regs . /Orders 

4.25 

4.53 

4.47 

4.83 

Military  Appearance 

4.53 

4.73 

4.75 

5.01 

Physical  Fitness 

4.51 

4.71 

4.74 

5.18 

Self-Control 

4.34 

4.49 

4.43 

4.40 

Leadership  Potential 

3.79 

4.20 

4.46 

4.65 

Supervisory  Ratings 

Technical  Skill 

4.84 

5.15 

5.18 

5.69 

Effort 

4.72 

5.02 

5.05 

5.53 

Following  Regs /Orders 

4.81 

5.08 

5.13 

5.62 

Military  Appearance 

4.72 

4.95 

5.09 

5.44 

Physical  Fitness 

4.66 

4.87 

5.02 

5.50 

Self-Control 

4.83 

5.07 

5.15 

5.57 

Leadership  Potential 

4.41 

4.78 

4.92 

5.51 

Training  Knowledge  Test 

104.35 

112.62 

110.90 

112.51 

^Means  on  the  ABLE  and  Spatial /Psychomotor  Composite  are  expressed  in 
terms  of  percentiles. 

Whe  Spatial /Psychomotor  score  is  a  weighted  composite  of  the  scores 
from  the  Spatial  and  Psychomotor  tests. 


Table  2:  Comparison  of  Means  on  Predictors  and  Criteria:  Application  of 
New  Decision  Rules  to  EIA  Graduates 


Variable 


Total  Equal  Emphasis  ABLE  Emphasis 

EIA  Grad.  Included  Excluded  Included  Excluded 

N  -  162  N  -  77  N  -  80  N  -  66  N  -  91 


Predictors 


AFQT 

64 

70 

59 

68 

62 

ABLE® 

57 

76 

39 

85 

35 

Spa/Psyc^ 

65 

79 

Criteria 

48 

66 

62 

Peer  Ratinss 

Technical  Skill 

5.23 

5.43 

5.05 

5.35 

5.14 

Effort 

4.41 

4.49 

4.34 

4.60 

4.27 

Follow  Regs . /Orders 

4.83 

4.93 

4.75 

4.83 

4.83 

Military  Appearance 

5.01 

5.05 

4.97 

5.00 

5.01 

Physical  Fitness 

5.18 

5.13 

5.22 

5.12 

5.22 

Self-Control 

4.40 

4.44 

4.37 

4.33 

4.45 

Leadership  Pot. 

4.65 

4.72 

4.59 

4.71 

4.61 

Sunervisory  Ratines 

Technical  Skill 

5.69 

5.89 

5.52 

6.02 

5.46 

Effort 

5.53 

5.77 

5.32 

5.75 

5.37 

Follow  Regs /Orders 

5.62 

5.72 

5.54 

5.88 

5.44 

Military  Appearance 

5.44 

5.64 

5.27 

5.68 

5.27 

Physical  Fitness 

5.50 

5.59 

5.43 

5.63 

5.41 

Self-Control 

5.57 

5.67 

5.48 

5.67 

5.50 

Leadership  Pot. 

5.51 

5.64 

5.40 

5.67 

5.40 

Trng.  Knowledge  Test 

112.51 

117.45 

108.33 

115.64 

110.27 

^Means  on  the  ABLE  and  Spatial /Psychomotor  Composite  are  expressed  in 
terms  of  percentiles. 

Whe  Spatial /Psychomotor  score  is  a  weighted  composite  of  the  scores 
from  the  Spatial  and  Psychomotor  tests. 
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Computing  Army  Aptitude  Area  Composite  Scores 
(1980  metric)  for  the  Armed  Services  Vocational 
Aptitude  Battery  (ASVAB) ,  Forms  6  and  7 


INTRODUCTION 

In  October  1984  the  reference  population  for  all  military 
aptitude  tests  was  changed  from  the  1944  mobilization  popula¬ 
tion  to  the  1980  youth  population.  At  that  time  four  new 
versions  of  the  operational  Armed  Service  Vocational  Battery 
(ASVAB  11/12/13/14)  and  a  single  form  of  the  in-service  test 
battery,  the  Armed  Forces  Classification  Test  (AFCT-9) ,  were 
issued  with  conversion  tables  appropriate  for  the  new  reference 
population. 

Since  only  one  form  of  the  in-service  test  is  currently  in 
use,  the  Army  has  no  viable  recourse  in  the  case  of  potential 
compromise  of  the  AFCT. 

The  last  form  of  the  AFCT,  prior  to  the  change  in 
reference  populations  was,  a  renormed  version  of  ASVAB  7B.  All 
forms  of  ASVAB  6/7  used  conversion  tables  based  on  the  1944 
mobilization  population  as  the  reference  population.  The 
purpose  of  this  research  is  to  produce  new  conversion  tables, 
appropriate  to  1980  youth  reference  population,  to  be  used  with 
AFCT-7B. 

BACKGROUND 

With  the  introduction  of  ASVAB  11/12/13/14  the  reference 
population  for  military  aptitude  tests  became  the  1980  youth 
population  of  all  17-23  year  old  youth  (both  male  and  female) . 
In  order  to  compare  scores  received  prior  to  October  1984  to 
those  received  afterward,  it  was  necessary  to  develop  a  method 
to  put  scores  on  a  common  reference  population  metric 
(preferably  the  1980  metric) .  Since  the  test  administered  to 
the  youth  population,  ASVAB  8A,  had  been  previously  normed  to 
the  1944  mobilization  population  (Maier  &  Grafton,  1980) , 
conversion  tables  between  standard  scores  in  the  two  metrics 
could  be  directly  computed  from  the  youth  population  data 
itself.  Tables  converting  AFQT  percentile  scores  as  well  as 
service-specific  aptitude  area  composite  standard  scores  from 
the  1944  metric  to  the  1980  metric  were  published  just  prior  to 
the  implementation  of  the  1980  metric  in  the  operational  ASVAB 
(Mitchell  and  Hanser,  1984;  Sims,  et.  al.,  1984). 

PROBLEM 

Generally  only  one  form  of  the  Armed  Forces  Classification 
Test  is  available  for  in-service  testing.  The  ASVAB  form  to  be 
used  is  selected  by  an  inter-service  committee.  The  choice  is 
made  among  forms  that  are  no  longer  in  operational  use.  The 
current  AFCT  is  the  only  ASVAB  form  from  the  ASVAB  8/9/10 


series  that  has  not  been  designated  for  another  specific 
purpose.  If  an  interim  replacement  for  the  current  AFCT  is  to 
be  used,  it  must  be  selected  from  the  ASVAB  6/7  series. 
Conversion  tables  for  the  recalibrated  versions  of  ASVAB  5/6/7 
were  published  in  July  1980  (Department  of  Defense,  1980) . 

These  tables  were  calibrated  to  1944  mobilization  population. 

In  order  to  use  any  form  of  ASVAB  6/7  as  an  alternate  form  of 
the  AFCT,  new  conversion  tables  need  to  be  developed  so  that 
scores  will  be  computed  in  the  1980  metric. 

METHOD 

The  computational  formulas  for  the  10  Army  Aptitude  Area 
Composites  are  from  the  revised  conversion  tables  for  ASVAB 
6/7/6E/7E  (Department  of  Defense,  1980) .  Table  A1  of  the 
appendix  lists  the  formulas  and  the  composites.  Raw  composite 
scores  for  ASVAB  6/7  are  sums  of  subtest  raw  scores.  Once  the 
sum  of  subtest  raw  scores  is  computed  for  each  aptitude  area 
score,  the  corresponding  aptitude  area  composite  standard  score 
is  obtained  by  a  table  look  up  procedure. 

Each  of  the  composite  standard  scores  in  Tables  1  and  2  of 
DoD  1304. 12W1  (Revised  July  1980),  was  converted  to  the  equiva¬ 
lent  1980  aptitude  area  composite  score  by  using  Table  A2  in 
Sims,  et.  al.  (1980).  Tables  A2  and  A3  of  the  Appendix  provide 
direct  conversions  for  the  ASVAB  6/7  raw  composite  scores  to 
1980-metric  aptitude  area  standard  scores.  The  original  1944- 
metric  are  not  provided  in  Tables  A2  and  A3  in  order  to  prevent 
errors  in  conversion  should  the  tables  be  used  in  actual 
scoring. 

The  Armed  Forces  Qualification  Test  (AFQT)  score  is  not 
recomputed  when  the  AFCT  is  taken  for  record.  For  this  reason, 
tables  for  converting  AFQT  scores  from  the  1944  metric  to  the 
1980  metric  are  not  provided.  These  tables  are  available, 
however,  in  both  Mitchell  and  Hanser  (1984)  and  in  Sims,  et. 
al.  (1984). 


DISCUSSION  AND  CONCLUSIONS 

If  the  Armed  Forces  Classification  Test  (AFCT)  needs  to  be 
replaced  due  to  compromise  of  the  current  version,  the  only 
possible  candidates  are  from  the  ASVAB  6/7  series.  AFCT  form 
7B  appears  to  be  a  likely  replacement.  If  AFCT-7B  or  any  other 
form  from  the  ASVAB  6/7  series  is  to  be  used  operationally,  the 
scores  should  be  reported  in  the  1980  metric.  Tables  Al  -  A3 
of  the  appendix  provide  a  procedure  for  accomplishing  this  goal 
and  should  be  issued  to  the  field  as  appropriate. 
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TABLE  A1 

COMPUTATION  FORMULAS® 

ARMY  APTITUDE  AREA  COMPOSITES 
ARMED  FORCES  CLASSIFICATION  TEST  (JAN  82) 


COMPOSITE 
General  Technical 
General  Maintenance 
Electronics 
Clerical 

Mechanical  Maintenance 

Surveillance/Communications 

Combat 

Field  Artillery 
Operators/Food 
Skilled  Technical 


FORMULA 

GT=AR+WK 

GM=AR+GS+MC+AI 

EL=AR+EI+MC+SI+CE 

CL=AR+WK+AD+CA 

MM=MK+SI+EI+AI+CM 

SC=AR+WK+MC+SP 

CO=AR+SI+SP+AD+CC 

FA=AR+GI+MK+EI+CA 

OF=GI+AI+CA 

ST=AR+MK+GS 


®  Composites  are  computed  as  sum  of  subtest  raw  scores. 
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TABLE  A2 

ARMY  TABLES  FOR  COMPOSITES 
ARMED  FORCES  CLASSIFICATION  TEST,  FORM  7B 

Composite  Standard  Scored 


Raw  Score 

CT 

GM 

EL 

105-110 

136 

104 

136 

103 

136 

102 

136 

101 

136 

100 

135 

99 

135 

98 

133 

97 

133 

96 

132 

95 

132 

94 

132 

93 

131 

92 

131 

91 

130 

90 

129 

89 

127 

88 

127 

87 

125 

86 

124 

85 

122 

84 

121 

83 

120 

82 

118 

81 

118 

80 

118 

79 

116 

78 

138 

115 

77 

136 

114 

76. 

136 

113 

75 

136 

113 

74  ' 

135 

111 

73 

134 

111 

72  - 

134 

110 

71 

133 

109 

70 

132 

107 

69  i 

132 

106 

68 

131 

105 

67 

130 

103 

6b 

129 

102 

CL 

MM 

Raw  Score 

137 

105-110 

136 

104 

136 

103 

135 

102 

135 

101 

129  1 

134 

100 

129 

134 

99 

129 

133 

98 

129 

132 

97 

129 

132 

96 

129 

131 

95 

129 

130 

94 

129 

129 

93 

129 

127 

92 

129 

127 

91 

129 

127 

90 

128 

126 

89 

128 

125 

88 

128 

124 

87 

128 

123 

86 

128 

123 

85 

127 

121 

84 

127 

121 

83 

125 

119 

82 

125 

119 

81 

125 

118 

80 

123 

117 

79 

122 

116 

78 

122 

116 

77 

121 

115 

76 

120 

115 

75 

118 

113 

74 

116 

113 

73 

115 

112 

72 

114 

110 

71 

114 

109 

70 

112 

109 

69 

112 

108 

68 

111 

107 

67 

108 

105 

66 

Raw  Score 

65 

64 

63 

62 

61 

60 

59 

58 

57 

56 

55 

54 

53 

52 

51 

50 

49 

48 

47 

46 

45 

44 

43 

42 

41 

40 

39 

38 

37 

36 

35 

34 

33 

32 

31 

30 

29 

28 

27 

26 
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TABLE  A2  (Continued) 
Composite  Standard  Score 


GT 

EL 

CL 

MM 

Raw  Score 

127 

101 

108 

104 

65 

124 

100 

108 

104 

64 

123 

99 

107 

102 

63 

122 

98 

106 

101 

62 

121 

97 

105 

100 

61 

119 

96 

103 

99 

60 

118 

96 

102 

98 

59 

118 

93 

102 

96 

58 

116 

92 

99  ! 

95 

57 

115 

91 

98 

95 

57 

115 

90 

97 

93 

55 

114 

89 

95 

92 

54 

114 

88 

93 

91 

53 

112 

88 

90 

89 

52 

112 

87 

89 

88 

51 

128 

111 

85 

89 

88 

50 

127 

109 

84 

86 

87 

49 

125 

108 

84 

85 

86 

48 

124 

107 

83 

84 

84 

47 

123 

105 

81 

82 

84 

46 

121 

103 

80 

81 

82 

45 

118 

102 

80 

79 

81 

44 

116 

101 

79 

77 

79 

43 

116 

99 

77 

74 

78 

42 

115 

98 

76 

72 

77 

41 

113 

97 

75 

70 

76 

40 

112 

96 

74 

68 

74 

39 

111 

94 

73 

67 

73 

38 

109 

92 

72 

64 

72 

37 

109 

91 

71 

62 

71 

36 

108 

89 

71 

61 

70 

35 

103 

87 

70 

59 

68 

34 

■:C2 

87 

68 

58 

67 

33 

101 

85 

67 

58 

66 

32 

19 

83 

67 

56 

64 

31 

•''7 

81 

64 

52 

63 

30 

'j’o 

80 

63 

50 

62 

29 

53 

79 

63 

49 

59 

28 

09 

77 

63 

48 

57 

27 

S 

76 

62 

48 

57 

26 

TABLE  A2  (Continued) 
Composite  Standard  Score 


1  October  1988 


Raw  Score 

GT 

GM 

EL 

CL 

Raw  Score 

25 

86 

74 

62 

48 

57 

25 

24 

83 

73 

62 

48 

57 

24 

23 

80 

71 

62 

48 

57 

23 

22 

78 

70 

62 

48 

57 

22 

21 

77 

69 

62 

48 

57 

21 

20 

75 

68 

62 

48 

57 

20 

19 

73 

67 

62 

48 

57 

19 

18 

71 

64 

62 

48 

57 

18 

17 

70 

60 

62 

48  ; 

57 

17 

16 

68 

60 

62 

48 

57 

16 

15 

66 

60 

62 

48 

57 

15 

14 

64 

60 

62 

48 

57 

14 

13 

61 

60 

62 

48 

57 

13 

12 

61 

60 

62 

48 

57 

12 

11 

58 

60 

62 

48 

57 

11 

0-10 

57 

60 

62 

48 

57 

0-10 

^All  standard 

scores 

are 

in  the 

1980  metric. 
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TABLE  A3 

ARMY  TABLES  FOR  COMPOSITES 
ARMED  FORCES  CLASSIFICATION  TEST,  FORM  7B 


Composite  Standard  Score^ 


Raw  Score 

SC 

CO 

FA 

OF 

ST 

Raw  Score 

110-117 

137 

110-117 

109 

137 

109 

108 

136 

108 

107 

135 

107 

106 

135 

106 

105 

135 

136 

f 

r 

105 

104 

133 

136 

104 

103 

133 

136 

103 

102 

133 

136 

102 

101 

133 

136 

101 

100 

133 

136 

100 

99 

133 

136 

99 

98 

132 

136 

98 

97 

132 

136 

97 

96 

130 

134 

96 

95 

129 

134 

95 

94 

128 

133 

94 

93 

127 

133 

93 

92 

126 

132 

92 

91 

125 

131 

91 

90 

132 

133 

130 

90 

89 

132 

122 

130 

89 

88 

132 

121 

129 

88 

87 

132 

119 

128 

87 

86 

131 

118 

128 

86 

85 

131 

117 

127 

85 

84 

129 

116 

125 

84 

83 

128 

115 

124 

83 

82 

127 

114 

124 

82 

81 

126 

113 

123 

81 

80 

125 

112 

125 

80 

79 

124 

111 

12^ 

79 

78 

123 

110 

119 

78 

77 

122 

109 

11^ 

77 

76 

120 

106 

lit 

76 

75 

119 

104 

lit 

75 

74 

118 

103 

114 

74 

73 

118 

102 

113 

73 

72 

117 

100 

115 

72 

71 

116 

99 

110 

71 
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TABLE  A3  (Continued) 


Composite  Standard  Score 


T  Score 

SC 

CO 

FA 

OF 

ST 

Raw  Score 

70 

115 

98 

110 

70 

69 

114 

96 

109 

69 

68 

113 

94 

108 

68 

67 

113 

93 

106 

67 

66 

112 

91 

105 

66 

65 

111 

89 

104 

65 

64 

111 

89 

102 

64 

63 

110 

87 

101 

f 

63 

62 

110 

86 

99 

62 

61 

108 

86 

99 

61 

60 

108 

83 

98 

135 

60 

59 

106 

82 

97 

135 

59 

58 

104 

81 

95 

135 

58 

57 

104 

80 

94 

133 

57 

56 

103 

79 

91 

132 

56 

55 

103 

77 

91 

134 

131 

55 

54 

101 

75 

90 

134 

129 

54 

53 

100 

74 

88 

134 

128 

53 

52 

99 

74 

87 

134 

126 

52 

51 

98 

71 

86 

134 

124 

51 

50 

96 

69 

84 

134 

123 

50 

49 

94 

69 

83 

134 

122 

49 

48 

93 

68 

83 

134 

121 

48 

47 

91 

67 

82 

134 

120 

47 

46 

90 

66 

81 

132 

117 

46 

45 

89 

65 

80 

130 

116 

45 

44 

87 

64 

78 

129 

115 

44 

43 

85 

62 

78 

129 

114 

43 

42 

85 

60 

75 

125 

113 

42 

41 

82 

60 

75 

124 

112 

41 

40 

81 

58 

74 

121 

ilo 

40 

39 

80 

58 

73 

119 

109 

39 

38 

78 

57 

72 

118 

109 

38 

37 

75 

57 

70 

116 

107 

37 

36 

74 

57 

70 

115  ‘ 

106 

36 

35 

71 

57 

69 

114 

104 

35 

34 

70 

57 

67 

111 

103 

34 

33 

68 

57 

66 

108 

102 

33 

32 

66 

57 

65 

107 

99 

32 

31 

64 

57 

64 

104 

98 

31 

1  October  1988 


Ray  Score 

30 

29 

28 

27 

26 

25 

24 

23 

22 

21 

20 

19 

18 

17 

16 

15 

14 

13 

12 

11 

0-10 

®A11  standard 


TABLE  A3  (Continued) 


Composite  Standard  Score 


SC 

CO 

FA 

OF 

Raw  Score 

63 

57 

63 

101 

97 

30 

61 

57 

62 

99 

94 

29 

59 

57 

59 

96 

92 

28 

56 

57 

58 

93 

90 

27 

53 

57 

58 

89 

89 

26 

50 

57 

58 

88 

86 

25 

49 

57 

58 

86 

85 

24 

48 

57 

58 

84  ; 

83 

23 

48 

57 

58 

81 

81 

22 

48 

57 

58 

77 

79 

21 

48 

57 

58 

75 

78 

20 

48 

57 

58 

71 

74 

19 

48 

57 

58 

68 

71 

18 

48 

57 

58 

66 

69 

17 

48 

57 

58 

63 

67 

16 

48 

57 

58 

61 

65 

15 

48 

57 

58 

58 

63 

14 

48 

57 

58 

55 

62 

13 

48 

57 

58 

53 

59 

12 

48 

57 

58 

53 

58 

11 

48 

57 

58 

53 

50 

0-10 

scores  are  in  the  1980  metric. 
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Belationship  between  Project  A  PsychcmotOT  and  ^ati^  Tests 
and  TCIN2  Gurmeiy  Peacfoonmanoes  A  PreLiininary  Investigation 

mmCDDCEIGN 

Within  the  Army,  matcMng  the  ri^t  person  to  the  ri^t  job  con^ues 
to  be  an  inportant  issue.  Ongoing  research  efforts  atterpt  to  optimize 
person-to-job  natches.  One  such  major  effcart  is  the  Aoty's  Project  A: 
Inproving  the  Selection,  Classification  and  UtilizatiOT  of  Amy  Enlteted 
Personnel.  One  of  Project  A's  principal  goeils  is  to  ittprove  traditional 
nethods  and  add  new  ones  to  enable  roeasurement  of  a  greater  range  of 
abilities  required  for  successful  soldier  performance  (Eaton,  Goer, 
Harris,  Si  Zook,  1984).  This  work  has  resulted  in  development  of  a 
ccnputerized  and  pencil— and— paper  best  battery  of  tenperament,  cognitive, 
psychcmotor,  perceptual  and  spatial  variables  vhich  are  ejpected  to 
predict  performance  in  a  diversity  of  Military  Occupational  Specialties 
(MDS) .  Early  results  indicate  that  the  Project  A  batb^  has  incremental 
vali^ty  in  predicting  enlisted  perfornenoe  over  that  obtained  ^th  the 
Anted  Services  Vocational  iptitude  Battery  (ASVAB)  alone,  the  single 
current  selection/classification  tool. 

As  work  on  Project  A  has  progressed,  it  has  beccme  apparent  that  there 
are  special  applications  for  v^ch  parts  of  the  battery  have 
considerable  potential.  Among  the  nost  apparent  is  the  use  of  psychcmo- 
tor  and  spatial  bests  for  the  selection  of  gunners.  Ihe  t^ts  have 
alreacfy  been  researched  with  M— 1  tank  gunners  using  the  Unit  Conduct  of 
Fire  Trainer  (UCOFT)  (Smith  &  Graham,  1987).  The  high  correlation 
between  the  tests  and  performance  (R  =  .76)  suggests  that  similar  resuilts 
with  other  gunnery  specialties  are  possible.  The  present  effort  extends 
research  on  the  validity  of  the  Project  A  tests  for  assigning  basic 
trainees  in  Infantry  into  Advanced  Individual  Training  (AIT)  for  MDS  IIH 
(TOW  gunner) .  The  work  was  undertaken  at  the  request  of  the  Ccirmanding 
General  of  the  Training  and  Doctrine  Ccmmand  (TRftDOC) . 


The  advent  of  anti-armor  ^bans  that  fire  wire-guided  missiles  for 
Infantry  use  brought  new  considerations  to  the  issues  of  personnel 
selection  and  training  (Gartner,  Strasel,  Evans,  Heller,  &  Tierney, 

1985).  Both  the  heavy  anti-tank  TOW  (Tube-launched,  Optically-tracked, 
Wire-guided)  and  weapons  of  the  "fire  and  forget"  category  such  as  the 
bazooka  and  M-1  tank  require  abilities  of  target  acquisition  and 
identification,  prediction  of  movanent  direction,  and  judgment  of  v*en  to 
fire.  The  TOW,  however,  also  requires  that  the  gunn^  maintain  the 
missile  on  target  until  the  missile  hits,  i.e.,  continuous  si^t 
alignment.  The  ground  mount  TCW2,  the  system  of  interest  in  this 
research,  requires  the  gunner  to  manipulate  knobs  with  bo^  ha]^  (one 
for  the  horizontal  axis  and  one  for  the  vertical  axrs)  while  visually 
tracking  the  target  through  an  infrared  optical  device.  The  gunner  must 
exert  considerable  motor  control  over  the  TCW  launcher  to  maintain  the 
sight  at  the  target's  center  and  iteke  both  gross  and  fine  motor 
adjustments  (Gartner,  et.  al,  1985). 


Ihese  coordination  and  tracking  skills  needed  ty  the  TOW  systeros  are 
not  readily  assessed  by  ASVZffl.  New  Project  A  tests  ocmputerized 
psycihcttDtor  tasks  and  paper-and-pencil  spatial  tests  have  been  developed 
to  measure  these  kinds  of  abilities.  This  research  was  undertaken  to  see 
if  an  assignment  procedure  that  includes  measures  of  psychonotor  and 
spatial  ability  would  be  likely  to  reduce  training  time  and  costs  and/or 
result  in  more  proficient  TCW  gunners. 

NEIHD 


Subjects  were  410  IIH  soldiers  vdio  participated  in  the  Project  A 
testing  during  the  Longitudinal  Validation  of  the  Predictor  Battery 
during  FY  87.  M-70  Qualification  Scores  were  cAxtainable  for  353  of  these 

soldiers.  A  srtaller  subsample  of  81  soldiers  ves  used  in  sane  of  the 
analyses.  These  were  soldiers  for  whan  additional  performance  data  were 
available.  These  data  inclixied  the  number  of  attempts  requir^  to  p^s 
each  of  eight  hands-on  measures  of  critical  tasks  tested  during  training 
in  addition  to  tracking. 

Description  of  the  ihstrumentg 

The  instruments  used  in  this  research  were  developed  to  measure 
cognitive,  psychonotor,  perceptual  and  spatial  abilities.  A  summary 
description  is  provided  in  Table  1.  Additional  information  is  provided 
below  and  in  Peterson  (1987). 

(INSERT  TABLE  1  ABOUT  HERE) 


Paper-and-Pencil  Tests 

RpHFKTnintT.  This  test  measures  the  ability  to  identic  a  logical  rule 
or  pattern  governing  a  series  of  figures  and  select  the  next  figure  in 
the  series  fran  five  cLLtematives . 

Cbiect  Rotatian.  This  test  measures  the  ability  mentally  to  rotate 
two-dimensional  figures  or  to  manipulate  their  ooiponent  parts.  The 
examinee  must  indicate  whether  an  item  figure  is  the  "same"  (rotated)  or 
"not  same"  (flipped)  as  the  stimulus. 

This  test  measures  ability  to  vistialize  hew 
cortponent  parts  will  fit  together  to  form  a  ccnpleted  object.  Each  test 
item  presents  a  picture  of  several  cenponents.  The  examinee  must  select 
the  one  response  out  of  four  that  shews  the  parts  correctly  put 
together. 

OHentatinn.  This  test  measures  the  ability  to  maintain  one's 
perspective  or  bearing  with  respect  to  some  object  when  it  and  its 
cenponent  parts  have  been  rotated.  Each  item  presents  a  non-v^jri^t 
picture  wdthin  a  frame  containing  a  circle  with  a  dot  at  the  bottom.  The 
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examinee  inentally  inust  rotate  the  frame  to  align  the  circle  vith  the 
bottcna  of  the  picture.  Ihen  he  must  determine  the  position  of  the  dot. 

Ifaze.  This  test  measures  ability  to  scan  a  ccnplex  field  visually  and 
identic  a  particular  pattern^  configuration,  or  pathway  within  it.  For 
each  item,  the  examinee  must  decide  \hich  of  four  labeled  (A  to  D) 
entrances  to  a  maze  leads  to  an  exit. 

Mrm.  Subjects  are  presented  with  naps  with  landrrerks  (e.g. ,  mess 
hall,  caicpsite,  lake)  but  no  ccnpass  directions.  Test  items  provide 
infomation  ab^t  direction  (e.g.,  "The  forest  is  north  of  the 
campsite " )  vhich  examinees  use  to  determine  direction  to  travel  to 
another  landmark. 

Onmniiter  Tests 

Overall.  The  ocnputers  used  a  specially  designed  response  pedestal 
pictured  in  Figure  1.  TVro  joysticks  allow  subjects  to  choose  right  or 
left  hand.  There  are  also  horizontal  and  vertical  slide  controls,  three 
response  buttons  (blue,  vhite,  and  yellow),  and  two  red  response  buttons. 
Four  green  "home"  buttons  ensiire  controlling  the  location  of  subjects' 
hands  during  stimulus  presentation,  controlling  the  onset  of  test  itanns, 
and  enabling  assessment  of  both  decision  time  and  movement  time  com¬ 
ponents  of  reaction  time. 

(Insert  Figure  1  about  here) 

Simple  Reaction  Tune.  The  first  five  items  of  this  test  are 
practice  items  to  enable  the  siibject  to  get  accustomed  to  the  equipment 
and  are  not  scored.  Subjects  start  with  hands  on  the  HOME  buttons  (reacty 
position) .  For  each  item,  the  subject  must  respond  as  quickly  as 
possible  when  the  word  "YELLCW"  appears  hy  pressing  the  yellow  response 
button.  To  eliminate  extreme  scores  caused  by  extraneous  factors,  the 
fastest  and  slowest  responses  are  deleted. 

Choice  Rmctim  Timp,  This  test  requires  a  choice  response.  Instead 
of  the  stimulus  "YELLCW",  the  stiraulvis  is  either  "BLUE"  or  "WHITE".  When 
it  appears,  the  subject  must  release  the  HCME  button  and  press  the 
correct  color  button. 

Msmorv.  This  tests  measures  how  quickly  individuals  can  use 
information  from  short-term  memory.  A  set  of  xp>  to  five  letters  briefly 
appears  on  the  screen  followed  by  presentation  of  a  single  probe  letter. 
The  task  is  to  decide  if  the  pzxte  letter  was  contained  in  the  original 
display  and  press  the  WHITE  (yes)  or  BLUE  (no)  button. 

One-Band  TcackinQ  .  This  pursuit  tracking  task  using  a  joystick 
requires  the  subject  to  keep  crosshairs  centered  on  a  target  box  as  it 
travels  at  a  constant  speed  along  a  path  with  both  horizontal  and 
vertical  segments.  Speed  of  crosshair  and  of  targets,  length  of  the 
path,  and  nimiber  of  segments  making  vp  the  path  vary  across  trials. 
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Peamtyhvil  Speed  &  Acxajacacy.  This  tests  requires  examinees  to  cxnpare 
rapidly  two  sets  of  up  to  9  characters  (letters,  numbers,  symbols,  car 
re^  words)  that  are  presented  siinultaneously.  Subjects  in^cate  pairs 
are  “same"  (WHITE  button)  or  "different"  (BEJUE  button) . 

iWfv-hand  Tcaddna.  This  is  another  pursuit  tracking  test,  but  it  is 
designed  to  measure  multilimb  (tvro-handed)  coordination.  The  subject 
controls  movement  of  the  crosshairs  using  the  vertical  and  horizontal 
slides  instead  of  a  joystick. 

WiwtTfyr  Mpmnrv.  This  tests  the  ability  to  do  sitiple  arithmetic 
operations  quickly  and  accurately.  Items  consist  of  sequential 
presentation  of  numerical  operations  until  an  answer  is  presented. 

Subject  presses  a  button  to  indicate  if  the  solution  is  correct  (WHITE) 
or  not  (BLUE). 

rantnnn  Phnot.  This  test  measures  ability  to  hit  a  target  moving  at 
constant  speed  and  direction  by  firing  from  a  stationcuy  position.  The 
location  of  the  cannon,  speed  and  direction  of  the  target,  angle  of 
target  to  cannon,  and  distance  from  cannon  to  inpact  point  and  from  fire 
point  to  impact  point  vary  across  trials. 

Target  Identification  Test.  This  test  requires  subjects  to  match 
target  figures  (sinple  renditions  of  militery  vdiicles  and  aircraft)  to 
one  of  three  stimulus  figures  labelled  BLUE,  YELDCW,  and  WHITE,  xising  the 
appropriate  buttons 

Target  Shoot  Test.  At  the  start  of  a  trial,  the  target  box  and 
crosshairs  appear  at  different  points  on  the  screen.  While  the  target 
moves  about  unpredictably,  changing  both  speed  and  direction,  the 
subject  \;>ses  the  joystick  to  center  the  caesshairs  on  the  target  and 
presses  a  RED  button  to  "fire". 

Criterion  VariaMPi« 

As  is  often  the  case  in  valicJation  research  in  an  applied  or 
qperaticmal  setting,  the  perfornance  critericn  was  of  primary  concern 
here.  An  adequate  criterion  must  be  reliable,  valid,  and  cxmtain 
sufficient  variance  to  differentiate  between  good  and  bad  performance. 

The  M-70  simulator  is  used  for  both  training  and  evaluaticsn  during  TCW 
gunner  AIT,  and  the  scores  cbtained  from  this  simulator  appear  to  meet 
these  requirements  (Hughes  Aerospace  Groi^),  1967).  All  TCW  gunners  must 
achieve  a  passing  score  on  the  M-70  simulator  prior  to  graduation.  FOr 
the  s\±>sample  of  ceases  for  v^cm  full  training  ciata  was  available,  hands- 
on  scx)res  for  eight  critical  taslcs  were  provided  as  well. 

The  M-70  simulator  requires  tracking  of  a  ireal  target  through  an 
infrared  cptical  device.  The  simulator  provides  iK)ise  effects 
cxitparable  to  a  "live  fire"  situation.  One  dravijack  to  the  M-70  training 
exercise  is  that  it  does  not  require  a  soldier  to  track  along  both 
vertical  and  horizcantal  axes,  i.e,  cm  ground  that  is  not  level;  instead, 
the  training  exercise  requires  a  soldier  to  track  a  target  that  is 
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novtng  at  a  cxanstant  speed/  in  a  horizcjntel  trajectory,  along  a  pay^, 
ijoad.  Biese  particular?  M— 70  data,  therefore,  may  l^ve  Ijinrted 
Qeneralizahility  to  other  more  demanding  tracking  situations. 

vL-nn  Scare.  The  M-70  score  is  a  measure  of  tracking 

behavior  over  a  series  of  simulated  TCW  firings.  The  score  for  each 
firing  reflects  the  probability  of  a  hit.  According  to  the  Huc^  ^ 
Aerospace  Group  (1967),  the  score  used  to  obtain  the  probability  esta^te 
is  analogous  to  the  standard  deviation  of  the  deviations  of  the  TOW  off 
of  its  track  during  its  trajectory.  The  scores  on  each  trial  are 

sunned  over  all  ten  trials  of  the  TCW  exercise.  The  minimum  score 
required  to  qualify  as  a  TCW  gunner  is  550  out  of  a  possible  1000. 

M-70  RaUTVT.  These  ratings  are  based  on  the  qualification  score.  The 
four  ratings  have  been  assigned  the  values  0-3  for  this  research  and  are 
as  follows:  0=UnquaUfied  (score  <  550),  l="2nd  Class"  (550-649),  2="lst 
Class"  (650-749),  3="E}qpert"  (750-1000).  Using  ^s  criterion  allowed 
the  incliasion  of  an  additional  17  soldiers  vho  did  not  qualify  and 
therefore  did  not  have  M-70  Qualification  Scores. 

TfaTvig-r>i  «vr»rpg.  The  Taw2  training  program  includes  instruction  in 
assanbling,  itaintaining,  and  firing  the  TCW  and  in  deployment  and  safety 
as  well  as  tracking.  Soldier  trainees  must  demonstrate  the  ability  to 
perform  eight  critical  tasks  vital  to  their  job  tefore  being  tested  m 
the  M-70  simulator.  To  demonstrate  proficiency  in  these  tasks,  soldiers 
must  receive  a  "Go"  \?on  actual  hands-on  performance  of  the  tasks.  These 
tasks,  as  listed  in  the  student  progress  chart,  are: 

1.  AssOTble  the  TCW2  launcher 

2.  Conduct  system  checkout  procedures  and  preoperational  inspection  of 
the  TCW2  launcher  and  encased  missile. 

3.  Perform  operator's  iraintenance  on  the  TCW2  launcher. 

4.  Load,  aim,  and  unload  an  encased  missile. 

5.  Perform  imrrediate  achion  procedures  for  a  TCW2  misfire. 

6.  Determine  if  a  target  is  engageable. 

7 .  Prepare  an  anti-armor  range  card  (TCW) . 

8.  E}q)lain  TCW  firing  limitations  and  characteristics. 

The  relaticjnship  between  these  iirportant  aspects  of  TW  gunnery 
perforitance,  in  acidition  to  tracking,  and  Project  A  predictors  was 
investigated  as  well.  For  this  research,  each  task  was  scared  as  the 
number  of  attenpts  required  to  achieve  a  "Go"  and  an  overall  sum  vras 
carputed.  POur  attempts  were  allowed  on  each  task;  fail\ire  to  achieve  a 
"Go"  was  scx>red  as  a  "5"  on  tasks  1—7 .  All  reported  scores  on  task  8 
were  "1" ;  the  few  cases  \here  task  8  was  unscared  were  also  cansicJered 
to  have  scored  a  "1".  Hands-on  scores  were  available  for  the  svhsaraple 
only. 

ErnoediireR 

prcH^r-i-ntr  -t-iagt-iwj.  Soldiecs  W6re  tested  on  the  Project  A  measures  in 
the  Reception  Station.  Approximately  half  the  subjects  tocik  the  ocmputer 
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tests  first  while  others  took  the  pencil-and-paper  tests  first.  All 
si±ijects  received  the  timed  pencil-and-paper  tests  in  the  order  indicated 
in  Table  1.  Subjects  vgere  able  to  ask  questions  after  the  instructions 
wei?e  read  to  them.  Ihe  ocirputerized  tests  were  self -administered  on 
Seequa  Chaiteleon  oatputers.  All  instructions  were  provided  on  the 
ootputer  screen.  A  test  administrator  \jas  present  during  testing  to 
handle  c[uestions  and/or  ec[uipinent  problems. 

fvitgrrifwi  Testlnci.  Scores  were  obtained  from  records  and  not  subject 
to  researcher  control.  Nontal  procedure  for  training  in  firing  and 
tracking  on  the  M-70  includes  four  days  of  training  uixier  different 
conditions  (e.g.,  ranges,  target  speeds, nic^t  tracking,  rapid  fire). 

Qiia  1 T  fi  ra-t-.i  nn  firing  on  the  fifth  day  involves  firing  in  the  M-70  fitted 
with  the  M-80  blast  simulator  (Gartner,  et.  al.,  1985). 

analygps 

General  descriptive  statistics  (means  and  standard  deviations)  were 
calculated  for  predictor  and  M-70  scores.  Correlations  between  the 
predictor  tests  and  criterion  scores  for  the  total  saitple  and  for  the 
subsanple  were  calculated  as  well. 

RESOEOS 

Means  and  standard  deviations  for  each  of  the  tests  used  from  the 
Project  A  Battery,  the  M-70  Qualification  Score,  and  the  ASVZffl  aptitude 
Area  00  (Cciibat)  score  for  the  total  saitple  and  for  the  subsairple  are 
provided  in  Tables  2  and  3  respectively.  A  catparison  of  the  means  in 
Tables  2  and  3  indicates  that  only  Perceptual  Speed  and  Accuracy  Decision 
Tiire  and  Target  Identification  Decision  Time  ireans  were  significantly 
different  between  the  two  groups  at  the  .05  level.  Because  two  of  the  33 
ccnparisons  would  be  expected  to  be  significant  by  chance  alone,  no 
further  effort  was  espended  in  group  ccrparisons. 

(Insert  Tables  2  and  3  about  here) 

Table  4  contains  the  correlations  between  the  Project  A  predictors 
and  the  criteria  for  both  the  sanple  and  the  subsanple.  The  two  criteria 
used  for  the  sanple  were  the  M-70  Qualification  sooa?e  and  the  M-70 
rating.  The  subsanple  also  has  correlations  ireported  for  the  total 
Hands-on  score.  The  Project  A  tests  that  significantly  predict  M-70 
Qualification  scores  and  ratings  in  both  sanples  are  Cne-Hand  Tracking, 
TWo-Hand  Tracking,  Reasoning  and  Orientation.  In  the  total  sanple,  the 
additional  significant  predictors  of  both  criteria  were  Object  Rotation 
and  Map.  In  the  subsanple,  the  Maze  test  predicted  all  three  criteria 
significantly.  In  general,  the  correlation  coefficients  are  in  the 
expected  direction.  Longer  response  times  and  greater  tracking  error 
distances  are  usually  indicative  of  poorer  performance. 

(Insert  Table  4  about  here) 
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Another  vsy  to  evaluate  the  potential  inpact  of  using  Project  A 
neasures  to  choose  soldiers  to  enter  gunner  training  is  to  ccnpare  the 
performance  of  soldiers  scoring  hic^i  on  the  Project  A  measures  with  the 
total  groip.  In  this  analysis,  the  total  sample  \as  divided  into  thirds 
on  the  Total  Tracking  Score,  the  sum  of  standardized  scores  for 
One-hand  and  l^ro-hand  tracking.  The  distrihuticn  of  M-70  ratings 
then  computed  for  those  scoring  in  the  top  third  of  the  ^tal  Tracking 
Scojce  as  well  as  for  the  entire  sample.  Figure  2  summarizes  the 
comparison  made  between  the  toro  groups.  In  the  total  sample,  8.9  percent 
of  the  soldiers  scored  as  Eb^perts  on  their  final  M-70  Qualification 
Score;  in  the  top  third  subsample,  the  percentage  of  Eb^jerts  increased  to 
12.9  percent.  Perhaps  a  more  important  difference  is  seen  for  those  \dio 
qualified  as  2nd  Class.  In  the  total  sample,  61.0  percent  sTOred  as  2nd 
Class  v^le  in  the  groip  having  "hi^"  Projert  A  Total  Tracking  scores 
only  48.3  percent  fell  in  this  lowest  qualified  category. 

(Insert  Figure  2  about  here) 

DISCOSSICN 

As  in  the  case  of  the  UOQET  research,  this  preliminary  research  into 
the  effectiveness  of  Project  A  psychcmotor  and  spatial  tests  in 
predicting  success  in  TCW  gunner  training  yields  premising  results. 

Using  one  or  itore  of  the  Project  A  tests  to  choose  Infantry  basic 
trainees  to  be  assigned  to  further  training  as  TCW  gunners  should  result 
in  more  proficient  gunners  at  a  relatively  small  added  cost  for  testing. 

Additional  analyses  need  to  be  performed  on  the  subsample  where 
additional  data  is  available  on  p^ormance  during  training.  These 

data  include  scores  on  individual  Hands-On  tasks  as  well  as  an 
initial  M-70  score  that  is  not  as  confounded  with  training  and  practice 
effects  as  is  the  M-70  Qualification  Score.  If  additional  training  data 
could  be  collected,  the  effectiveness  of  the  Project  A  tests  in 
predicting  failures  could  cilso  be  evaluated. 

The  neset  st^  in  this  research  will  be  to  determine  vhich  of  the 
Project  A  tests  could  add  both  independent  and  significant  validity  to 
the  assignment  process  for  TOW  gunners.  Althou^  the  current  cenputer 
and  spatial  test  battery  requires  approximately  two  and  one-half  to  three 
hours  to  administer,  a  reduced  battery  of  two  to  four  tests  found  to  be 
sufficient  for  TOW  gunners  could  be  administered  in  less  than  one  hour. 
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Table  1 


Smmnarv  of  Project  A  Predictor  Meaeures 


Teat 

Number 

Time 

Hame 

Conatruct 

of  Itema 

Limit 

Score 

Reliability 

Paper  and  Fencll  Teata 


Reaaoning  Teat 

Induction 

30 

12  min* 

No*  correct 

.87" 

Object  Rotation 

Spatial  Viauali- 

90 

7.5  min* 

No*  correct 

•  99 

Teat 

aationt  Rotation 

Orientation 

Spatial 

2A 

10  min* 

No*  correct 

.89 

Teat 

Orientation 

Maze  Teat 

Spatial  Viauali- 
aationt  Scanning 

24 

5*5  min* 

No*  correct 

*96 

Map  Teat 

Spatial 

Orientation 

20 

12  min* 

No.  correct 

.90 

Aaaembling  Objecta 

Spatial  Viauali- 
sationt  Rotation 

32 

26  min. 

No*  correct 

.91 

Computerized  Teata 

Simple  Reaction 

Proceaaing 

15 

Oeciaion  time  mean 

*88 

Time 

Efficiency 

Movement  time  mean 

— 

X  correct 

*46 

Choice  Reaction 

Proceaaing 

30 

I.*. 

Oeciaion  time  mean 

*97 

Time 

Efficiency 

Movement  time  mean 

— 

X  correct 

.57 

Memory  Teat 

Short  Term  Memory 

36 

Oeciaion  time  mean 

*96 

Movement  time  mean 

— 

X  correct 

*60 

Target  Tracking  1 

Paychomotor 

18 

«... 

Mean  log 

•  98 

Preciaion 

(diatance  1) 

Perceptual  Speed 

Perceptual  apeed 

36 

Oeciaion  time  mean 

.94 

and  Accuracy 

and  accuracy 

Movement  time  mean 

— 

Z  correct 

*65 

Target  Tracking  2 

Two-hand 

18 

Mean  log 

*98 

coordination 

(diatance  +  1) 

Number  Memory 

Number  Operationa 

28 

Pinal  reaponae 

*88 

Teat 

time  mean 

Input  reaponae 
time  mean 

*95 

Pooled  operation 

V 

mean 

*93" 

Z  correct 

*59 

Cannon  Shoot 

Movement  judgment 

36 

••• 

Abaolute  time 

diacrepancy  Mean 
Mean  log 
(diatance  4  1) 

*65 

error 

-- 

Z  hita 

-- 

Target 

Perceptual  apeed 

36 

Oeciaion  time  mean 

*97 

Identification 

and  accuracy 

Movement  time  mean 

— 

Z  correct 

.62 

Target  Shoot 

Paychomotor 

30 

Mean  log 

Teat 

preciaion 

(diatance  +1) 

error 

.74 

Mean  time  to  fire 

.85 

Z  hita 

— 

^Split-half  (odd-even)  reliability  with  Spearman-Brown  Correction  for  length  baaed  on 
Project  A  Concurrent  Validation  (CV)  data  ■  9332  to  93A5). 

"Coefficient  alpha  reliability  baaed  on  CV  data* 


Table  2 

Total  Sample  Means  and  Standard  Deviations  for  ASVAB  CO  Aptitu^ 
Area  Composite,  the  Project  A  Tests  and  M70  Qualification  Score 


N* 

Mean'* 

S.D. 

ASVAB 

CO  Composite 

406 

113.5 

12.4 

Simple  Reaction  Time 

Percent  Correct 

Decision  Time  Mean 

Movement  Time  Mean 

409 

409 

409 

98 

30.11 

25.64 

06 

11.36 

16.61 

Choice  Reaction  Time 

Percent  Correct 

Decision  Time  Mean 

Movement  Time  Mean 

410 

410 

410 

98  . 
38.47 
24.29 

04 

8.03 

6.90 

Short  Term  Memory 

Percent  Correct 

Decision  Time  Mean 

Movement  Time  Mean 

406 

406 

406 

89 

77.33 

34.08 

07 

21.00 

11.66 

One-hand  Tracking 

Mean  Log  (Dist  +  1)  Error 

408 

2.66 

.38 

Perceptual  Speed  and.  Accuracy 
Percent  Correct 

Decision  Time  Mean 

Movement  Time  Mean 

407 

407 

408 

86 

210.85 

30.81 

09 

60.87 

10.63 

Two-hand  Tracking 

Mean  Log  (Dist  +1)  Error 

408 

3.30 

.47 

Number  Memory 

Percent  Correct 

Initial  Response  Time  Mean 
Pooled  Operations  Time  Mean 
Final  Response  Time  Mean 

404 

404 

404 

404 

88 

13.20 

20.18 

137.52 

10 

42.2 

7.63 

37.00 

Cannon  Shoot 

Absolute  Time  Descrepancy  Mean 

403 

41.04 

7.45 

Target  Identification 

Percent  Correct 

Decision  Time  Mean 

Movement  Time  Mean 

398 

398 

398 

90 

154.74 

33.78 

07 

51.81 

9.20 

Target  Shoot 

Mean  Time  To  Fire 

Mean  Log  (Dist  +  Error) 

393 

393 

208.27 

2.08 

.22 

45.36 

Reasoning 

Object  Rotation 

Orientation 

Maze 

Map 

Assembling  Objects 

390 

390 

390 

390 

390 

390 

20.7 

63.1 
13.0 
18.4 

9.1 

25.2 

5.1 

19.4 

6.5 
4.3 

5.6 

6.6 

353 

651.8 

differ  due  to  mleelng  data 
^Tlmee  are  in  hundredths  of  seconds 


Table  3 

Subsample  Means  and  Standard  Deviations  for  ASVAB  CO,, Aptitude 
Area  Compofiites.  Project  A  Tests  and  M70  Qualification  Sco£e 


H*  Mean^  S.D. 


ASVAB 

CO  Composite 

79 

111.4 

11.4 

Simple  Reaction  Time 

Percent  Correct 

Decision  Time  Mean 

Movement  Time  Mean 

81 

81 

81 

98 

28.79 

23.84 

05 

6.31 

11.00 

Choice  Reaction  Time 

Percent  Correct 

Decision  Time  Mean 

Movement  Time  Mean 

81 

81 

81 

98 

37.29 

24.07 

03 

6.28 

7.92 

Short  Term  Memory 

Percent  Correct 

Decision  Time  Mean 

Movement  Time  Mean 

80 

80 

80 

90 

74.00 

31.66 

06 

16.54 

9.65 

One-Hand  Tracking 

Mean  Log  (Dist  +  1)  Error 

80 

2.69 

.42 

Perceptual  Speed  and  Accuracy 
Percent  Correct 

Decision  Time  Mean 

Movement  Time  Mean 

80 

80 

80 

85 

195.77 

28.80 

10 

58.68 

8.55 

Two-Hand  Tracking 

Mean  Log  (Dist  +1)  Error 

80 

3.29 

.4'8 

Number  Memory 

Percent  Correct 

Initial  Response  Time  Mean 

Pooled  Operations  Time  Mean 

Final  Response  Time  Mean 

79 

79 

79 

79 

88 

13.09 

18.74 

132.50 

10 

4.64 

7.07 

32.48 

Cannon  Shoot 

Absolute  Time  Descrepancy  Mean 

77 

40.84 

6.90 

Target  Identification 

Percent  Correct 

Decision  Time  Mean 

Movement  Time  Mean 

76 

76 

76 

89 

143.51 

32.91 

08 

41.71 

8.64 

Target  Shoot 

Mean  Time  To  Fire 

Mean  Log  (Dist  +  Error) 

72 

72 

209.30 

2.06 

.17 

50.02 

Reasoning 
object  Rotation 

Orientation 

Maze 

Map 

Assembling  Objects 

80 

80 

80 

80 

80 

80 

20.6 

63.0 

13.7 

18.6 

9.1 

26.1 

5.3 

19.2 

5.9 

4.2 

5.3 

5.4 

M70  Oualification  Score 

81 

640.6 

70.7 

•Ns  differ  due  to  Bissing  data 
^Times  are  in  hundredths  of  seconds 


Tabl«  4 


Zero-Order  Correlatione  Between  Predictor  Varlnble  and  M»70  Criterion  Seorae. 


Pr.dlctor.  _ Crit.rU _ _ _ _ _ 

Total  Sample 

Sub sample^ 

M70  Qual.*  M70** 

Scores  Rating 

H70  Qual. 
Scores 

M70 

Rating 

Hands  on 
Total 

Simple  Reaction  Time 

Percent  Correct  -*01 

Decision  Time  Mean  -.02 

Movement  Time  Mean  00 

Choice  Reaction  Time 

Percent  Correct  00 

Decision  Time  Mean  *05 

Movement  Time  Mean  -.07 

Short  Term  Memory 

Percent  Correct  -.07 

Decision  Time  Mean  *05 

Movement  Time  Mean  -.02 

Tracking  1 

Mean  Log  (Diet  +  1)  Error  -.18*** 

Perceptual  Speed  and  Accuracy 

Percent  Correct  -Ol 

Decision  Time  Mean  -.07 

Movement  Time  Mean  *01 

Tracking  2 

Mean  Log  (Diet  +U  Error  -.28*** 

Humber  Memory 

Percent  Correct  *05 

Initial  Response  Time  Mean  -.06 

Pooled  Operations  Time  Mean  -.02 

Pinal  Response  Time  Mean  -.01 

Cannon  Shoot 

Absolute  Time  Descrepancy  Mean  -.15** 

Target  Identification 

Percent  Correct  -.02 

Decision  Time  Mean  -.18*** 

Movement  Time  Mean  *02 

Target  Shoot 

Mean  Time  To  Eire  -.11 

Mean  Log  (Diet  +  Error)  -.10 

Reasoning  No.  Correct  .14* 

Object  Rotation  Ho.  Correct  .17** 

Orientation  No.  Correct  .23*** 

Maze  Ho.  Correct  *08 

Map  Ho.  Correct  .16** 

Assembling  Objects _ *06 


-.01 

.04 

.17 

.07 

-.02 

.10 

.07 

.13 

00 

-.14 

-.18 

-.05 

-.03 

-.05 

-.05 

-.08 

-.01 

.19 

•  15 

.01 

-.10* 

-.12 

-.19 

-.03 

.03 

.06 

.00 

-.19 

•  00 

.16 

.20 

.09 

-.08 

-.24* 

-.20 

.03 

-.18*** 

-.41*** 

-.34** 

.31** 

-.02 

.02 

.01 

-.18 

-.06 

-.09 

-.16 

-.09 

-.04 

-.16 

-.17 

-.02 

-.27*** 

-.40*** 

-.35** 

.  .29** 

.05 

•  19 

.17 

-.21 

-.08 

-.11 

-.11 

•  05 

-.03 

.01 

-.01 

.08 

-.02 

-.02 

•  00 

.07 

-.13* 

-.22 

-.24* 

.12 

-.04 

-.10 

-.09 

.05 

-.16** 

-.18 

-.19 

.04 

-.02 

-.04 

-.01 

.04 

-.16** 

-.40*** 

-.37** 

•  20 

-.12* 

-.22 

-.16 

.27* 

.16** 

.28* 

.29** 

-.24* 

.20*** 

.12 

.15 

-.20 

.28*** 

.33** 

.37*** 

-.34*** 

.14** 

.32** 

.29** 

-.38** 

.21*** 

.20 

.15 

-.14 

.11* 

.17 

.11 

-.31** 

^ns  range  from  338-353 
**n8  range  from  355-370 
^ns  range  from  72-81 

*p  <  .05;  **p  <  .01;  ***p  <  .001 


Figure  1.  Response  pedestal  for  computerized  tests. 
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Relationship  Between  Project  A  Psychomotor  and  Spatial  Tests 
and  T0W2  Gunnery  Performance:  A  Preliminary  Investigation 

INTRODUCTION 

Within  the  Army,  matching  the  right  person  to  the  right  job  continues  to  be 
an  important  issue.  Ongoing  research  efforts  attempt  to  optimize  person-to-job 
matches.  One  such  major  effort  is  the  Army’s  Project  A:  Improving  the 
Selection,  Classification  and  Utilization  of  Army  Enlisted  Personnel.  One  of 
Project  A’s  principal  goals  is  to  improve  traditional  methods  and  add  new  ones 
to  enable  measurement  of  a  greater  range  of  abilities  required  for  successful 
soldier  performance  (Eaton,  Goer,  Harris,  &  Zook,  1984).  This  work  has 
resulted  in  development  of  a  computerized  and  pencil-and-paper  test  battery  of 
temperament,  cognitive,  psychomotor,  perceptual  and  spatial  variables  which  are 
expected  to  predict  performance  in  a  diversity  of  Military  Occupational 
Specialties  (MOS).  Early  results  indicate  that  the  Project  A  battery  has 
incremental  validity  in  predicting  enlisted  performance  over  that  obtained  with 
the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  alone,  the  single 
current  selection/classification  tool. 

As  work  on  Project  A  has  progressed,  it  has  become  apparent  that  there  are 
special  applications  for  which  parts  of  the  battery  have  considerable 
potential.  Among  the  most  apparent  is  the  use  of  psychomotor  and  spatial  tests 
for  the  selection  of  gunners .  The  tests  have  already  been  researched  with  M-1 
tank  gunners  using  the  Unit  Conduct  of  Fire  Trainer  (UCOFT)  (Smith  &  Graham, 
1987).  The  high  correlation  between  the  tests  and  UCOFT  performance  (R  »  .76) 
suggests  that  similar  results  with  other  gunnery  specialties  are  possible.  The 
present  effort  extends  research  on  the  validity  of  the  Project  A  tests  for 
assigning  basic  trainees  in  Infantry  into  Advanced  Individual  Training  (AIT) 
for  MOS  IIH  (T0W2  gunner).  The  work  was  undertaken  at  the  request  of  the 
Commanding  General  of  the  Training  and  Doctrine  Command  (TRADOC). 

Background 

The  advent  of  anti-armor  systems  that  fire  wire-guided  missiles  for  Infantry 
use  brought  new  considerations  to  the  issues  of  personnel  selection  «nd 
training  (Cartner,  Strasel,  Evans,  Heller,  &  Tierney,  1985).  Both  the  heavy 
(Tube-launched,  ^tically- tracked.  Wire-guided)  and  weapons  of 
the  "fire  and  forget"  category  such  as  the  bazooka  and  M-1  tank  require 
abilities  of  target  acquisition  and  identification,  prediction  of  movement 
direction,  and  judgment  of  when  to  fire.  The  T0W2,  however,  also  requires  that 
the  gunner  maintain  the  missile  on  target  until  the  missile  hits,  i.e., 
continuous  sight  alignment.  The'  ground  mount  T0W2,  the  system  of  interest  in 
this  research,  requires  the  gunner  to  manipulate  knobs  with  both  hands  to  move 
a  ong  a  vertical  axis  while  visually  tracking  the  target  through  an  infrared 
optical  device.  The  T0W2  is  swiveled  on  its  ground  mount  to  move  the  missile 
left  or  right.  The  gunner  must  exert  considerable  motor  control  over  the  T0W2 
launcher  to  maintain  the  sight  at  the  target’s  center  and  make  both  gross  and 
fine  motor  adjustments  (Cartner,  et.  al,  1985). 
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These  coordination  and  tracking  skills  needed  by  the  TOW2  system  are  not 
readily  assessed  by  ASVAB,  New  Project  A  computerized  psychomotor  tasks  and 
paper-and-pencil  spatial  tests  have  been  developed  to  measure  these  kinds  of 

This  research  was  undertaken  to  see  if  an  assignment  procedure  that 
includes  measures  of  psychomotor  and  spatial  ability  would  be  likely  to  reduce 
training  time  and  costs  and/or  result  in  more  proficient  T0¥2  guimers. 

METHOD 


Sample 

Subjects  were  410  IIH  soldiers  who  participated  in  the  Project  A  testing 
during  the  Longitudinal  Validation  of  the  Predictor  Battery  during  FY  87.  M-70 

Qualification  Scores  were  obtained  for  353  of  these  soldiers.  A  smaller 
subsample  of  81  soldiers  was  used  in  some  of  the  analyses.  These  were  soldiers 
for  whom  additional  performance  data  were  available.  These  additional  data 
included  the  number  of  attempts  required  to  pass  each  of  eight  hands-on 
measures  of  critical  tasks  tested  during  training  and  final  M-70  scores  for 
soldiers  who  did  not  gradu^O  form^AIT. 

Description  of  the  InstnimpTif-a  ' 

The  instruments  used  in  this  research  were  developed  to  measure  cognitive, 
psychomotor,  perceptual  and  spatial  abilities.  A  summary  description  is 

provided  in  Table  1.  Additional  information  is  provided  below  and  in  Peterson 
(1987) . 


(Insert  Table  1  about  here) 


Paper-and-Pencll  Testa 

Spatial  Reasoning.  This  test  measures  the  ability  to  identify  a  logical 
rule  or  pattern  governing  a  series  of  figures  and  select  the  next  figure  in  the 
series  from  five  alternatives. 

pb I ect_  Rotation.  This  test  measures  the  ability  mentally  to  rotate 
two-dimensional  figures  or  to  manipulate  their  component  parts.  The  examinee 
must  indicate  whether  an  item  figure  is  the  "same"  (rotated)  or  "not  same" 
(flipped)  as  the  stimulus. 

Assembling  Objects.  This  test  measures  the  ability  to  visualize  how  com¬ 
ponent  parts  will  fit  together  to  form  a  completed  object.  Each  test  item 
presents  a  picture  of  several  coAponents.  The  examinee  must  select  the  one 
response  out  of  four  that  shows  the  parts  correctly  put  together. 

prientation.  This  test  measures  the  ability  to  maintain  one’s  perspective 
or  bearing  with  respect  to  some  object  when  it  and  its  component  parts  have 
been  rotated.  Each  item  presents  a  non-upright  picture  within  a  frame 
containing  a  circle  with  a  dot  at  the  bottom.  The  examinee  mentally  must 
rotate  the  frame  to  align  the  circle  with  the  bottom  of  the  picture.  Then  he 
must  determine  the  position  of  the  dot. 
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Maze.  This  test  measures  the  ability  to  scan  a  complex  field  visually  and 
identify  a  particular  pattern,  configuration,  or  pathway  within  it.  For  each 
item,  the  examinee  must  decide  which  of  four  labeled  (A  to  D)  entrances  to  a 
maze  leads  to  an  exit. 

l^.  Subjects  are  presented  with  maps  containing  landmarks  (e.g.,  mess 
hall,  campsite,  lake)  but  no  legend  to  identify  compass  directions.  Test 
items,  however,  provide  information  about  direction  (e.g.,  "The  forest  is  north 
of  the  campsite")  which  examinees  use  to  determine  direction  to  travel  of 
another  landmark. 

Computer  Tests 

Overall .  The  computers  used  a  specially  designed  response  pedestal  pictured 
in  Figure  1.  Two  joysticks  allow  subjects  to  choose  right  or  left  hand.  There 
are  also  horizontal  and  vertical  slide  controls,  three  response  buttons  (blue, 
white,  and  yellow),  and  two  red  response  buttons.  Four  green  "home"  buttons 
ensure  controlling  the  location  of  subjects’  hands  during  stimulus  presenta¬ 
tion,  controlling  the  onset  of  test  items,  and  enabling  assessment  of  both 
decision  time  and  movement  time  components  of  reaction  time. 

(Insert  Figure  1  about  here) 

Simple  Reaction  Time .  The  first  five  items  of  this  test  are  practice  items 
to  enable  the  subject  to  get  accustomed  to  the  equipment  and  are  not  scored. 
Subjects  start  with  hands  on  the  HOME  buttons  (ready  position).  For  each  item, 
the  subject  must  respond  as  quickly  as  possible  when  the  word  "YELLOW"  appears 
by  pressing  the  yellow  response  button.  To  eliminate  extreme  scores  caused  by 
extraneous  factors,  the  fastest  and  slowest  responses  are  deleted. 

^Choice  Reaction  Time .  This  test  requires  a  choice  response.  Instead  of  the 
stimulus  "YELLOW",  the  stimulus  is  either  "BLUE"  or  "WHITE".  When  it  appears, 
the  subject  must  release  the  HOME  button  and  press  the  correct  color  button. 

Memory.  This  test  measures  how  quickly  individuals  can  use  information  from 
short-term  memory.  A  set  of  up  to  five  letters  briefly  appears  on  the  screen 
followed  by  presentation  of  a  single  probe  letter.  The  task  is  to  decide  if 
the  probe  letter  was  contained  in  the  original  display  and  press  the  WHITE 
(yes)  or  BLUE  (no)  button. 

One-Hand  Tracking  .  This  pursuit  tracking  task,  using  a  joystick,  requires 
the  subject  to  keep  crosshairs  centered  on  a  target  box  as  it  travels  at  a 
constant  speed  along  a  path  with 'both  horizontal  and  vertical  segments.  Speed 
of  crosshairs  and  of  targets,  length  of  the  path,  and  number  of  segments  making 
up  the  path  vary  across  trials. 

Perceptual  Speed  &  Accuracy .  This  test  requires  examinees  to  compare  two 
simultaneously-presented  sets  of  stimulus  arrays  with  up  to  9  characters  each 
(letters,  numbers,  symbols,  or  real  words).  Subjects  indicate  whether  the  two 
sets  are  "same"  (WHITE  button)  or  "different"  (BLUE  button). 
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Two-hand  Trackine.  This  is  another  pursuit  tracking  test,  but  it  is 
designed  to  measure  multilimb  (two-handed)  coordination.  The  subject  controls 
movement  of  the  crosshairs  using  the  vertical  and  horizontal  slides  instead  of 
a  joystick. 

Number  Memory.  This  test  measures  the  ability  to  do  simple  arithmetic 
operations  quickly  and  accurately.  Items  consist  of  sequential  presentation  of 
numerical  operations  until  an  answer  is  presented.  Subject  presses  a  button  to 
indicate  if  the  answer  is  correct  (WHITE)  or  not  (BLUE) . 

Cannon  Shoot.  This  test  measures  the  ability  to  hit  a  target  moving  at 
constant  speed  and  direction  by  firing  from  a  stationary  position.  The 
location  of  the  cannon,  speed  and  direction  of  the  target,  angle  of  target  to 
cannon,  and  distance  from  cannon  to  impact  point  and  from  fire  point,  to  impact 
point  vary  across  trials. 

Identification  Test .  This  test  requires  subjects  to  match  target 
figures  (simple  renditions  of  military  vehicles  and  aircraft)  to  one  of  three 
stimulus  figures  labelled  BLUE,  YELLOW,  and  WHITE,  using  the  appropriate 
buttons . 

Target  Shoot  Test.  At  the  start  of  a  trial,  the  target  box  and  crosshairs 
appear  at  different  points  on  the  screen.  While  the  target  moves  about 
unpredictably,  changing  both  speed  and  direction,  the  subject  uses  the  joystick 
to  center  the  crosshairs  on  the  target  and  presses  a  RED  button  to  "fire"  when 
the  crosshairs  rendezvous  with  the  target  box. 

Criterion  Variables 

As  is  often  the  case  in  validation  research  in  an  applied  or  operational 
setting,  the  performance  criterion  was  of  primary  concern  here.  An  adequate 
criterion  must  be  reliable,  valid,  and  contain  sufficient  variance  to 
differentiate  between  good  and  bad  performance.  The  M-70  simulator  is  used  for 
both  training  and  evaluation  during  T0W2  gunner  AIT,  and  the  scores  obtained 
from  this  simulator  appear  to  meet  these  requirements  (Hughes  Aerospace  Group, 
1967).  All  T0W2  gunners  must  achieve  a  passing  score  on  the  M-70  simulator 

graduation.  Again,  for  the  subsample  of  cases  for  whom  full  training 
data  was  available,  hands-on  scores  for  eight  critical  tasks  were  provided  as 
well. 

The  M-70  simulator  requires  tracking  of  a  real  target  through  an 
infrared  optical  device.  The  simulator  provides  noise  effects  comparable  to  a 
live  fire"  situation.  One  drawback  to  the  M-70  training  exercise  is  that  it 
does  not  require  a  soldier  to  track  along  both  vertical  and  horizontal  axes, 
i.e,  on  ground  that  is  not  level;  Instead,  the  training  exercise  requires  a 
soldier  to  track  a  target  that  is  moving  at  a  constant  speed,  in  a  horizontal 
trajectory,  along  a  paved,  level  road.  These  particular  M-70  data,  therefore, 
may  have  limited  generalizability  to  other  more  demanding  tracking  situations. 

Qualification  Score.  The  M-70  score  is  a  measure  of  tracking  behavior 
over  a  series  of  simulated  T0W2  firings.  The  score  for  each  firing  reflects 
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the  probability  of  a  hit.  According  to  the  Hughes  Aerospace  Group  (1967),  the 
score  used  to  obtain  the  probability  estimate  is  analogous  to  the  standard 
deviation  of  the  deviations  of  the  T0W2  off  of  its  ideal  track  during  its 
trajectory.  The  scores  on  each  trial  are  summed  over  all  ten  trials  of  the 
T0W2  exercise.  The  minimum  M-70  score  rec[ulred  to  qualify  as  a  T0W2  gunner  is 
550  out  of  a  possible  1000. 

M-70  Rating.  These  ratings  are  based  on  the  qualification  score.  The  four 
ratings  have  been  assigned  the  values  0-3  for  this  research  and  are  as  follows: 
0=Unqualified,  l='"2nd  Class"  (550-649),  2“"lst  Class"  (650-749),  3="Expert" 
(750-1000) . 

Hands-on  Scores.  The  T0¥2  training  program  includes  instruction  in  assem¬ 
bling,  maintaining,  and  firing  the  T0W2  and  in  deployment  and  safety  as  well  as 
tracking.  Soldier  trainees  must  demonstrate  the  ability  to  perform  eight 
c^^itical  tasks  vital  to  their  job  as  well  as  achieve  a  passing  score  on  the  M- 
70  simulator  before  they  can  graduate.  These  tasks,  as  listed  in  the  student 
progress  chart,  are: 

1.  Assemble  the  T0W2  launcher 

2.  Conduct  system  checkout  procedures  and  preoperational  inspection  of 

the  T0¥2  launcher  and  encased  missile. 

3.  Perform  operator’s  maintenance  on  the  T0¥2  launcher. 

4.  Load,  aim,  and  unload  an  encased  missile. 

5.  Perform  immediate  action  procedures  for  a  T0¥2  misfire. 

6.  Determine  if  a  target  is  engageable. 

7.  Prepare  an  anti-armor  range  card  (T0¥2). 

8.  Explain  T0¥2  firing  limitations  and  characteristics. 

The  relationship  between  these  important  aspects  of  T0¥2  gunnery  perform¬ 
ance,  in  addition  to  tracking,  and  Project  A  predictors  was  investigated  as 
well.  For  this  research,  each  task  was  scored  as  the  number  of  attempts 
required  to  achieve  a  "Go"  and  an  overall  sum  was  computed.  Four  attempts  were 
allowed  on  each  task;  failure  to  achieve  a  "Go"  was  scored  as  a  "5"  on  tasks 
1-7.  All  reported  scores  on  task  8  were  "1";  the  few  cases  where  task  8  was 
unscored  were  also  considered  to  have  scored  a  "1".  Hands-on  scores  were 
available  for  the  subsample  only. 

Procedures 


Predictor  testing.  Soldiers  were  tested  on  the  Project  A  measures  in  the 
Reception  Station  prior  to  being  selected  for  training  in  MOS  IIH.  Approxi¬ 
mately  half  the  subjects  took  the  computer  tests  first  while  others  took  the 
pencil-and-paper  spatial  tests  first.  All  subjects  received  the  timed 
pencil-and-paper  tests  in  the  order  indicated  in  Table  1.  Subjects  were  able 
to  ask  questions  after  the  Instructions  were  read  to  them.  The  computerized 
tests  were  self-administered  on  Seequa  Chameleon  computers.  All  instructions 
were ^ provided  on  the  computer  screen.  A  test  administrator  was  present  during 
testing  to  handle  questions  and/or  equipment  problems. 
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Criterion  Testing.  Scores  were  obtained  from  records  and  not  subject  to 
researcher  control.  Normal  procedure  for  training  in  firing  and  tracking  on 
the  M-70  includes  four  days  of  training  under  different  conditions  (e.g,, 
ranges,  target  speeds,  night  tracking,  rapid  fire).  Qualification  firing  on  the 
fifth  day  involves  firing  in  the  M-70  fitted  with  the  M-80  blast  simulator 
(Gartner,  et,  al.,  1985). 

Analyses 

General  descriptive  statistics  (means  and  standard  deviations)  were 
calculated  for  predictor  and  M-70  scores.  Correlations  between  the  predictor 
tests  and  criterion  scores  for  the  total  sample  and  for  the  subsample  were 
calculated  as  well. 


RESULTS 

Means  and  standard  deviations  for  each  of  the  tests  used  from  the  Project  A 
Battery,  the  M-70  Qualification  Score,  and  the  ASVAB  Aptitude  Area  CO  (Combat) 
score  for  the  total  sample  and  for  the  subsample  are  provided  in  Tables  2  and  3 
respectively.  A  comparison  of  the  means  in  Tables  2  and  3  indicates  that  only 
Perceptual  Speed  and  Accuracy  Decision  Time  and  Target  Identification  Decision 
Time  means  were  significantly  different  between  the  two  groups  at  the  .05 
level.  Because  two  of  the  33  comparisons  would  be  expected  to  be  significant 
by  chance  alone,  no  further  effort  was  expended  in  group  comparisons. 

(Insert  Tables  2  and  3  about  here) 

Table  4  contains  the  correlations  between  the  Project  A  predictors  and  the 
for  both  the  total  sample  and  the  subsample.  The  two  criteria  used 
for  the  sample  were  the  M-70  Qualification  score  and  the  M-70  rating.  The 
subsample  also  has  correlations  reported  for  the  total  Hands-on  score.  The 
Project  A  tests  that  significantly  predict  M-70  Qualification  scores  and 
ratings  in  both  samples  are  One-Hand  Tracking,  Two-Hand  Tracking,  Spatial 
Reasoning  and  Orientation.  In  the  total  sample,  the  additional  significant 
predictors  of  both  criteria  were  Object  Rotation  and  Map.  In  the  subsample, 
the  Maze  test  predicted  all  three  criteria  significantly.  In  general,  the 
correlation  coefficients  are  in  the  expected  direction.  Longer  response  times 
and  greater  tracking  error  distances  are  usually  indicative  of  poorer  perform¬ 
ance. 


(Insert  Table  4  about  here) 

As  can  be  seen  in  Table  4  there  is  an  apparent  difference  in  magnitude  of 
the  correlations  between  the  two  data  sets,  with  the  correlations  in  the 
smaller  data  set  being  larger  than  in  the  full  data  set.  The  differences 
between  correlations  in  the  two  data  sets  were  found  to  be  significant  (p<_.05) 
for  One-handed  Tracking,  Target  Shoot,  and  the  Maze  tests  (McNemar,  1969). 

Table  5  presents  the  correlations  with  M-70  Qualification  Scores,  ^s  and  the 
standard  error  of  the  difference  between  the  ^s  for  these  tests. 

(Insert  Table  5  about  here) 
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The  small  data  set  differed  slightly  from  the  total  data  set.  As  mentioned 
M-70  scores  were  available  for  all  soldiers  in  the  small  data  set 
including  the  three  soldiers  who  failed  to  qualify.  For  the  rest  of  the  total 
data  set,  however,  no  M-70  scores  were  received  for  soldiers  who  failed  the 
course.  There  were  a  total  of  19  failures  (Including  the  three  soldiers  from 
the  subsample).  All  M-70  Qualification  Scores  for  these  soldiers  were  set  to 
missing,  and  they  were  included  only  in  the  analysis  of  the  M-70  rating  scores. 
In  analyes  of  the  subsample,  the  reported  M-70  scores  were  retained  for  the 
three  course  failures. 

Among  the  three  failures  in  the  subset  was  a  significant  outlier  whose  final 
M-70  score  was  312.  The  mean  M-70  score  for  the  three  failures  was  502  as 
compared  to  646  for  the  other  78  soldiers  in  the  subsample.  In  order  to 
determine  the  possible  effect  the  extremely  low  M-70  score  might  have  had  upon 
the  correlations  in  the  subsample  the  lowest  M-70  score  (312)  was  recoded  to 
the  next  lowest  score(550)  and  the  correlations  were  recomputed.  Table  6 
compares  the  correlations  computed  using  the  original  scores,  the  scores  with 
the  outlier  recoded  and  the  total  sample  M-70  Qualification  scores.  ¥ith  the 
exception  of  the  Object  Rotation  test  all  of  the  correlations  decreased  in  the 
subsample  with  the  outlier  on  the  M-70  score  recoded  to  the  next  higher  score. 
The  correlations  shown  in  Table  5  as  being  significantly  different  from  the 

total  sample  are  no  longer  significantly  different  using  the  recoded  score  (see 
Table  6 ) . 


For  the  total  sample,  however,  M-70  qualification  scores  were  available  only 
for  those  soldiers  who  qualified  as  T0W2  gunners.  The  M-70  Qualification  score 
analyses  were  therefore  "restricted"  to  graduates  only  since  no  scores  were 
available  for  non-graduates.  An  additional  analysis  was  performed  on  the  total 
sample  after  assigning  all  non-graduates  an  M-70  score  of  549  (the  highest 
possible  non-qualifying  score.)  The  results  of  this  analysis  are  presented  in 
Table  7  for  those  predictors  included  in  Table  6.  As  can  be  sseen  in  Table  7, 
the  correlation  for  the  total  sample  increase  for  all  predictors  when  scores 
were  assigned  to  the  non-graduates. 

The  results  of  the  two  additional  analyses  show  that  the  two  sets  of 
correlations  are  much  more  similar  when  adjustments  are  made  for  known  dis¬ 
crepancies  in  the  data  sets  (extreme  outlier  in  the  subsample  and  assigning 
failing  M-70  scores  to  course  failures  in  the  total  sample.)  If  additional 
data  from  student  progress  charts  could  be  collected,  the  effect  of  outliers 
would  be  reduced  and  actual  final  M-70  scores  would  provide  a  more  acceptable 
criterion. 

Another  way  to  evaluate  the  potential  impact  of  using  Project  A  measures  to 
choose  soldiers  to  enter  gunner  training  is  to  compare  the  performance  of 
soldiers  scoring  high  on  the  Project  A  measures  with  the  total  group.  In  this 
analysis,  the  total  sample  was  divided  into  thirds  based  on  the  Total  Tracking 
Score,  the  sum  of  standardized  scores  for  One-hand  and  Two-hand  tracking.  The 
distribution  of  M-70  ratings  was  then  computed  for  those  scoring  in  the  top 
third  of  the  Total  Tracking  Score  as  well  as  for  the  entire  sample.  Figure  2 
summarizes  the  comparison  made  between  the  two  groups.  In  the  total  sample. 
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8.9  percent  of  the  soldiers  scored  as  Experts  on  their  final  M-70  Qualification 
Score;  in  the  top  third  subsample,  the  percentage  of  Experts  increased  to  12.9 
percent.  Perhaps  a  more  important  difference  is  seen  for  those  who  qualified 
as  2nd  Class.  In  the  total  sample,  61.0  percent  scored  as  2nd  Class  while  in 
the  group  having  "high"  Project  A  Total  Tracking  scores  only  48.3  percent  fell 
in  this  lowest  qualified  category. 

(Insert  Figure  2  about  here) 

DISCUSSION 

As  in  the  case  of  the  UCOFT  research,  this  preliminary  research  into  the 
effectiveness  of  Project  A  psychomotor  and  spatial  tests  in  predicting  success 
in  T0W2  gunner  training  yields  promising  results.  Using  one  or  more  of  the 
Project  A  tests  to  choose  Infantry  basic  trainees  to  be  assigned  to  further 
training  as  T0W2  gunners  should  result  in  more  proficient  gunners  at  a 
relatively  sniall  added  cost  for  testing. 

Additional  analyses  need  to  be  performed  on  the  subsample  where  additional 
data  is  available  on  performance  during  training.  These  additional  data 
include  scores  on  individual  Hands-On  tasks  as  well  as  an  initial  M-70  score 
that  is  not  as  confounded  with  training  and  practice  effects  as  is  the  M-70 
Qualification  Score.  If  additional  training  data  could  be  collected,  the 
effectiveness  of  the  Project  A  tests  in  predicting  failures  could  also  be 
evaluated . 

The  next  step  in  this  research  will  be  to  determine  which  of  the  Project  A 
tests  could  add  both  independent  and  significant  validity  to  the  current 
assignment  process  for  T0W2  gunners.  Although  the  full  computer  and  spatial 
test  battery  requires  approximately  two  and  one-half  to  three  hours  to  admin¬ 
ister,  a  reduced  battery  of  two  to  four  tests  found  to  be  sufficient  for  T0W2 
gunners  could  be  administered  in  less  than  one  hour. 
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Table  1 

Summary  of  Project  A  Predictor  Measures 


Test 

Name 

Construct 

Number 
of  Items 

Time 

Limit 

Score  Reliability 

Paper  and  Pencil  Tests 

Reasoning  Test 

Induction 

30 

12  min. 

No.  correct 

.87" 

Object  Rotation 

Test 

Spatial  Visuali¬ 
zation:  Rotation 

90 

7.5  min. 

No.  correct 

.99 

Orientation  Test 

Spatial  Orientation 

24 

10  min. 

No.  correct 

.89 

Maze  Test 

Spatial  Visuali¬ 
zation:  Scanning 

24 

5.5  min. 

No.  correct 

.96 

Map  Test 

Spatial  Orientation 

20 

12  min. 

No.  correct 

.90 

Assembling  Objects 

Spatial  Visuali¬ 
zation:  Rotation 

32 

26  min. 

No.  correct 

.91 

Computerized  Tests 

Simple  Reaction 

Time 

Processing 

Efficiency 

15 

— 

Decision  time  mean 
Movement  time  mean 

Z  correct 

.88 

.46 

Choice  Reaction 

Time 

Processing 

Efficiency 

30 

— 

Decision  time  mean 
Movement  time  mean 

Z  correct 

.97 

.57 

Memory  Test 

Short  Term  Memory 

36 

Decision  time  mean 
Movement  time  mean 

Z  correct 

.96 

.60 

Target  Tracking  1 

Psychomotor 

Precision 

18 

— 

Mean  log 
(distance  +1) 

.98 

Perceptual  Speed 
and  Accuracy 

Perceptual  Speed 
and  Accuracy 

36 

— 

Decision  time  mean 
Movement  time  mean 

Z  correct 

.94 

.65 

Target  Tracking  2 

Two-hand 

coordination 

18 

— 

Mean  log 
(distance  +1) 

.98 

Number  Memory 

Test 

Number  Operations 

28 

Final  response 
time  mean 

Input  response 
time  mean 

Pooled  operation 
mean 

Z  correct 

.88 

.95 

.93^* 

.59 

Cannon  Shoot 

Movement  judgment 

36 

Absolute  time 
discrepancy  Mean 

Mean  log 
(distance  1} 
error 

Z  hits 

.65 

Target 

Identification 

Perceptual  speed 
and  accuracy 

36 

— — 

Decision  time  mean 
Movement  time  mean 

Z  correct 

.97 

•  62 

Target  Shoot 

Test 

Psychomotor 

precision 

30 

Mean  log 
(distance  +1) 
error 

Mean  time  to  fire 

Z  hits 

.74 

.85 

Split-half  (odd-even)  reliability  with  Spearman -Brown  Correction  for  length  based  on 
Project  A  Concurrent  Validation  ((^)  data  (n  -  9332  to  9345). 

^Coefficient  alpha  reliability  based  on  CV  data. 
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Table  2 


Total  Sample  Means  and  Standard  Deviations  for  ASVAB  CO  Aptitude 
Area  Composite,  the  Project  A  Tests  and  M70  Qualification  Score 


N« 

Mean** 

S.D. 

ASVAB 

CO  Composite 

409 

113.5 

12.4 

Simple  Reaction  Time 

Percent  Correct 

Decision  Time  Mean 

Movement  Time  Mean 

409 

409 

409 

97.52 

30.11 

25.64 

5.74 

11.36 

16.61 

Choice  Reaction  Time 

Percent  Correct 

Decision  Time  Mean 

Movement  Time  Mean 

410 

410 

410 

98.03 

38.47 

24.29 

3.66 

8.03 

6.90 

Short  Term  Memory 

Percent  Correct 

Decision  Time  Mean 

Movement  Time  Mean 

406 

406 

406 

89.36 

77.33 

34.08 

6.59 

21.00 

11.66 

One -hand  Tracking 

Mean  Log  (Dist  +  1)  Error 

408 

2.66 

.38 

Perceptual  Speed  and  Accuracy 
Percent  Correct 

Decision  Time  Mean 

Movement  Time  Mean 

407 

407 

407 

85.65 

210.85 

30.81 

8.69 

60.87 

10.63 

Two-hand  Tracking 

Mean  Log  (Dist  +1)  Error 

408 

3.30 

.47 

Number  Memory 

Percent  Correct 

Initial  Response  Time  Mean 

Pooled  Operations  Time  Mean 

Final  Response  Time  Mean 

404 

404 

404 

404 

99.12 

13.20 

20.18 

137.52 

9.57 

4.22 

7.63 

37.00 

Cannon  Shoot 

Absolute  Time  Descrepancy  Mean 

403 

41.04 

7.45 

Target  Identification 

Percent  Correct 

Decision  Time  Mean 

Movement  Time  Mean 

398 

398 

398 

90.06 

154.74 

33.78 

7.20 
51.81 

9.20 

Target  Shoot 

Mean  Time  To  Fire 

Mean  Log  (Dist  +  Error) 

393 

393 

208.27 

2.08 

45.36 

.22 

Reasoning 

Object  Rotation 

Orientation 

Maze 

Map 

Assembling  Objects 

389 

393 

389 

386 

386 

386 

20.67 

63.06 

12.98 

18.41 

9.08 

25.18 

5.10 

19.43 

6.47 

4.30 

5.66 

6.61 

M70  Qualification  Score 

353 

651.81 

67.56 

*Ns  differ  due  to  missing  data 
**Times  are  in  hundredths  of  seconds 
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Table  3 


Subsample  Means  and  Standard  Deviations  for  ASVAB  CO  Aptitude 
Area  Composites.  Project  A  Tests  and  M70  Qualification  Score 


N! 

Mean** 

S.D. 

ASVAB 

CO  Composite 

79 

111.40 

11.44 

Simple  Reaction  Time 

Percent  Correct 

Decision  Time  Mean 

Movement  Time  Mean 

81 

81 

81 

97.92 

28.79 

23.84 

5.30 

6.31 
11.00 

Choice  Reaction  Time 

Percent  Correct 

Decision  Time  Mean 

Movement  Time  Mean 

81 

81 

81 

98.22 

37.29 

24.07 

2.72 

6.28 

7.92 

Short  Term  Memory 

Percent  Correct 

Decision  Time  Mean 

Movement  Time  Mean 

80 

80 

80 

89.72 

74.00 

31.66 

6.17 

16.54 

9.65 

One -Hand  Tracking 

Mean  Log  (Dist  +1)  Error 

80 

2.69 

.42 

Perceptual  Speed  and  Accuracy 
Percent  Correct 

Decision  Time  Mean 

Movement  Time  Mean 

80 

80 

80 

84.55 

195.77 

28.80 

9.79 

58.68 

8.55 

Two-Hand  Tracking 

Mean  Log  (Dist  +1)  Error 

80 

3.28 

.48 

Number  Memory 

Percent  Correct 

Initial  Response  Time  Mean 

Pooled  Operations  Time  Mean 

Final  Response  Time  Mean 

79 

79 

79 

79 

87.81 

13.09 

18.74 

132.50 

9.84 

4.64 

7.07 

32.48 

Cannon  Shoot ^ 

Absolute  Time  Descrepancy  Mean 

77 

40.84 

6.90 

Target  Identification 

Percent  Correct 

Decision  Time  Mean 

Movement  Time  Mean 

76 

76 

76 

88.51 

143.51 

32.91 

7.57 

41.71 

8.64 

Target  Shoot 

Mean  Time  To  Fire 

Mean  Log  (Dist  +  Error) 

72 

72 

209.30 

2.06 

50.02 

.17 

Reasoning 

Object  Rotation 

Orientation 

Maze 

Map 

Assembling  Objects 

80 

80 

80 

80 

80 

80 

20.66 

62.98 

13.68 

18.60 

9.11 

26.06 

5.33 

19.21 

5.93 

4.19 

5.27 

5.42 

M70  Qualification  Score 

81 

640.61 

70.67 

®Ns  differ 
'’Times  are 


due  to  missing  data 
in  hundredths  of  seconds 
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Table  4 


Zero«-Order  Correlations  Between  Predictor  Variable  and  M-70  Criterion  Scores 


Criteria 

Predictors 

Total  Sample 

Sub  sample^ 

M70  (Jual.® 
Scores 

Rating 

M70  Qaal. 
Scores 

M70 

Rating 

Hands  on 
Total 

Simple  Reaction  Time 

Percent  Correct 

-.01 

-.01 

.04 

.17 

.07 

Decision  Time  Mean 

-.02 

-.02 

.10 

.07 

.13 

Movement  Time  Mean 

.00 

.00 

-.14 

-.18 

-.05 

Choice  Reaction  Time 

Percent  Correct 

-.00 

-.03 

-.05 

-.05 

-.08 

Decision  Time  Mean 

.05 

-.01 

.19 

.15 

.01 

Movement  Time  Mean 

-.07 

-.10* 

-.12 

-.19 

-.03 

Short  Term  Memory 

Percent  Correct 

-.07 

-.03 

•  06 

.00 

-.19 

Decision  Time  Mean 

.05 

-.00 

.16 

.20 

-09 

Movement  Time  Mean 

-.02 

-.08 

-.24* 

-.20 

.03 

Tracking  1 

Mean  Log  (Diet  +  1)  Error 

-.18*** 

-.18*** 

-.41*** 

-.34** 

.31** 

Perceptual  Speed  and  Accuracy 

Percent  Correct 

.01 

-.02 

.02 

.01 

-.18 

Decision  Time  Mean 

-.07 

-.06 

-.09 

-.16 

-.09 

Movement  Time  Mean 

.01 

-.04 

-.16 

-.17 

-.02 

Tracking  2 

Mean  Log  (Dist  +1)  Error 

-,28*** 

-.27*** 

-.40*** 

-.35** 

.29** 

Number  Memory 

Percent  Correct 

-.05 

-.05 

.19 

.17 

-.21 

Initial  Response  Time  Mean 

-.06 

-.08 

-.11 

-.11 

.05 

Pooled  Operations  Time  Mean 

-.02 

-.02 

.01 

-.01 

.08 

Final  Response  Time  Mean 

-.01 

-.03 

-.02 

-.00 

.07 

Cannon  Shoot 

Absolute  Time  Descrepancy  Mean 

-.15** 

-.13* 

-.22 

-.24* 

.12 

Target  Identification 

Percent  Correct 

-.02 

-.04 

-.10 

-.09 

.05 

Decision  Time  Mean 

-.18*** 

-.16** 

-.18 

-.19 

.04 

Movement  Time  Mean 

.02 

-.02 

-.04 

-.01 

.04 

Target  Shoot 

Mean  Time  To  Fire 

-.11 

-.16** 

-.40*** 

-.37** 

.20 

Mean  Log  (Dist  +  Error) 

-.10 

-.12* 

-.22 

-.16 

.27* 

Reasoning  No,  Correct 

•  .14* 

.16** 

.28* 

,29** 

-.24* 

Object  Rotation  No.  Correct 

.17** 

.20*** 

.12 

.15 

-.20 

Orientation  No ,  Correct 

.23*** 

,28*** 

.33** 

.37*** 

-.34*** 

Maze  No,  Correct 

.08 

.14** 

.32** 

.29** 

-.38** 

Map  No ,  Correct 

.16** 

.21*** 

.20 

.15 

-.14 

Assembling  Objects 

.07 

.11* 

.17 

.11 

-.31** 

*118  range  from  338-353 
bns  range  from  355-370 
*n8  range  from  72  -  81 


*p  <  .05;  **p  <  ,01;  ***p  <  .001 
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Table  5 


.SlRnif leant  Differences  In  Correlations  between  Project  A  Tests  and 
M-70  Qualification  Scores  for  the  Two  Data  Sets 


Test 

Total 

r 

Subsample 

r 

z 

Tracking  1 

.18 

.44 

2.21 

.129 

Target  Shoot 

.11 

.40 

2.69 

.135 

Maze 

.08 

.32 

2.10 

.130 
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Table  6 


^parlaon  of  Correlations  Obtained  Using  Original  and  Recoded  Data  In  thr. 


Total  Sample 

Original 

Recoded 

Predictors 

M-70  Qoal 

Subsample 

Subsample 

Tracking 


Tracking  Composite 

-.25*** 

-.43*** 

-.33*** 

One -hand  Tracking 

-.18*** 

-.41*** 

-.30*** 

1.32 

Two-hand  Tracking 

-.28*** 

-.40*** 

-.33*** 

Target  Shoot 

Time  to  Fire 

-.11* 

-.40*** 

-.30*** 

1.84 

Distance 

-.10 

-.22 

-.14 

Cannon  Shoot 

Absolute  Mean  Disc.  Time 

-.15** 

-.22 

-.19 

Spatial  Tests 

Spatial  Reasoning 

.14* 

.28* 

.25* 

Object  Rotation 

.17** 

.12 

.15 

Orientation 

.23*** 

.33** 

.31** 

Maze 

.08 

.32** 

.26* 

1.39 

Map 

.16** 

•  20 

.19 

Assembling  Objects 

.06 

.17 

.11 

*p<_.05;  **p<_.01;  ***p<_.001 
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Table  7 


Comparison  of  Correlation  Obtained  Using  Original  and  Recoded  Data  in  the  Total  Sample 


-Predictors _ Total  Sample 


Recoded 

Subsample 

(n-81) 

Original 

M-70  Scores 
(n-353) 

With  Failure  M-70 

Scores  Added 
(n-374) 

Trackine 

Tracking  Composite 

-.33** 

-.25*** 

-.28*** 

One -Hand  Tracking 

-.30** 

-.18*** 

-.21*** 

Two-Hand  Tracking 

-.33** 

-.28*** 

-.29*** 

Target  Shoot 

Time  to  Fire 

-.30** 

-.11 

-.15** 

Distance 

-.14 

-.10 

-.11* 

Cannon  Shoot 

Absolute  Mean  Disc.  Time 

-.19 

-.15** 

-.16** 

Spatial  Test 

Spatial  Reasoning 

.25* 

.14* 

.15* 

Object  Rotation 

.15 

.17** 

,21*** 

Orientation 

.31*** 

,23*** 

.28*** 

Maze 

.26* 

.08 

.13* 

Map 

.19 

.16** 

,20*** 

Assembling  Oblects 

.11 

.06 

.10 

*P  <_  -05;  <_  .01;  ***p  <_  .001 

“Lowest  score  recoded  to  next  lowest  score. 
^Scores  for  all  failures  recoded  to  549. 
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Figure  1.  Response  pedistal  for  computerized  tests. 


Percent  Change  In  Qualifications 


Figure  1.  Expected  increase  in  qualified  TOW  gunners  because  of 
psychomotor  screening  (L.  Hanser,  1988). 
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Introduction 


The  purpose  of  this  paper  is  to  provide  a  detailed  response 
to  questions  about  Project  A  raised  by  GEN  Thurman  durinq  a 
briefing  at  Ft.  Benjamin  Harrison  on  13  July  1987.  GEN  Thurman 
requested  ART  to  provide  to  him^  by  NOS,  the  complete  data  on  the 
prediction  of  job  performance  using  the  ASVAB  test.  He  expressed 
interest  in  seeing  the  relationships  between  AFQT  category  and 
MOS-specific  task  level  performance  for  each  of  the  Project  A 
MOS. 


A  broad  overview  of  Project  A  is  provided  in  Appendix  A,  and 
copies  of  the  slides  from  the  13  July  1987  briefing  are  contained 
in  Appendix  B.  In  addition  to  the  specific  analyses  requested  by 
GEN  Thurman,  we  have  also  provided  summaries  of  several  analyses 
which  provide  for  greater  understanding  of  these  data. 

Background 

The  information  reported  in  this  paper  is  based  on  data  ^ 
collected  during  the  concurrent  validation  (CV)  phase  of  Project 
A.  In  the  summer  and  fall  of  1985  we  tested  approximately  9500 
first  tour  soldiers,  with  a  target  sample  size  of  600  per  MOS  in 
our  19  MOS.  The  19  MOS  are  listed  on  page  7  of  Appendix  B.  These 
MOS  were  divided  into  two  groups,  with  the  primary  distinction 
that  hands-on  tests  were  developed  for  the  first  group  of  nine 
MOS. 


In  addition  to  a  four-hour  battery  of  potentially  useful  new 
predictor  tests  to  augment  ASVAB  in  initial  entry  screening,  we 
administered  tests  to  soldiers  in  each  of  nine  MOS  in  order  to 
obtain  three  kinds  of  data  on  technical  job  performance.  These 
were;  (1)  hands-on  tests  (HO)  for  approximately  15  tasks,  (2) 
task-based  job  knowledge  tests  (K5)  for  these  15  tasks  and  an 
additional  15  tasks,  and  (3)  multiple-choice  training-based  job 
knowledge  items  in  a  number  of  sxibtests  (K3)  .  In  these  nine  MOS, 
performance  testing  took  12  hours  over  a  day  and  a  half.  For 
these  nine  MOS,  the  average  number  of  soldiers  tested  was  448. 
These  are  referred  to  as  the  Batch  A  MOS. 

For  soldiers  in  our  remaining  10  MOS  we  collected  the 
predictor  tests  and  training-based  job  knowledge  information 
only.  In  these  MOS  all  testing  took  place  in  one  day.  These  10 
MOS  are  referred  to  as  the  Batch  Z  MOS.  All  tests  were 
coordinated  with  and  approved  by  the  MOS  proponent  before  the 
data  collection. 
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Primary  Analyses 


Relationships  Between  AFOT  and  Task  Performance 

In  Appendix  C  are  tables  for  soldiers  in  each  AFQT  category 
showing  the  mean  percent  correct  or  mean  score  on  each  task  for 
each  MOS  in  our  sample.  Task  titles  preceded  by  "HO"  were  tested 
hands-on  while  those  preceded  by  "K5"  were  procedural  knowledge 
paper-and-pencil  tasks.  We  also  proyide  the  mean  score  for  all 
19  MOS  on  categories  of  the  training-based  subtests;  these  are 
designated  with  "K3"  on  the  printouts. 

We  haye  marked  each  task  or  subtest  with  an  "*"  if  there  is 
an  oyerall  statistically  significant  (p  <  .05)  increase  in 
performance  as  a  function  of  AFQT  category.  Within  those  tasks 
demonstrating  an  oyerall  significant  difference  we  haye 
additionally  marked  the  adjacent  groups  (II  ys.  IIIA,  IIIA  vs. 
IIIB,  and  IIIB  ys.  IV)  showing  a  significant  difference.  Due  to 
the  generally  small  sample  sizes  in  Category  I,  no  significance 
tests  were  computed  for  Category  I  vs.  Category  II,  and  we  did 
not  include  Category  I  soldiers  in  the  overall  significance 
tests.  Of  the  655  tasks  overall,  performance  on  59%  of  the  tasks 
increased  with  AFQT.  Across  the  19  MOS  the  lowest  number  of 
tasks  showing  a  significant  difference  across  AFQT  categories  was 
33%  for  MOS  91A,  and  the  highest  was  93%  for  54E  MOS.  Caution 
should  be  exercised  in  the  interpretation  of  cells  with  few 
soldiers. 

In  addition  to  the  individual  tasks  of  each  type  of  measure, 
several  aggregate  scores  are  also  analyzed  for  each  MOS.  For  all 
19  MOS  the  training-based  (K3)  subtests  are  aggregated  into  three 
to  six  scores  related  to  general  job  dimensions  (Communications, 
Vehicle  Maintenance/Operation,  Basic  Soldiering,  Identify  Target, 
Technical,  and  Safety).  Overall  MOS-specific  scores  for  each 
type  of  task  administered  (HO,  K5,  and  K3),  total  MOS-specific 
Core  Technical  Proficiency  and  General  Soldiering  Proficiency 
scores  are  analyzed  whenever  available.  We  were  also  able  to 
merge  these  data  with  the  SQT  data  file,  and  have  shown  for  each 
MOS  (except  54E  and  91A)  the  mean  SQT  score  for  soldiers  in  each 
AFQT  category. 

In  general,  although  some  individual  tasks  may  not  show  a 
difference  across  AFQT  categories,  the  total  scores  for  each  type 
of  task  are  significant  for  all  MOS.  The  exception  to  this  is 
the  hands-on  component  for  the  nine  Batch  A  MOS.  The  number  of 
hands-on  tasks  showing  significant  differences  among  AFQT 
categories  varies  from  a  low  of  one  out  of  fifteen  for  MOS  63B 
(seven  percent)  to  a  high  of  six  out  of  seventeen  (thirty-five 
percent)  for  MOS  13B  and  95B.  Five  of  the  Batch  A  MOS  (IIB,  13B, 
31C,  71L  and  91A)  show  significant  differences  for  the  total 
Hands-On  Performance  score  (HO  Core  Technical  Proficiency)  but 
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the  other  four  (19E,  63B,  64C,  and  95B)  do  not.  As  a  basis  for 
comparison,  the  number  of  job  knowledge  (K5)  tasks  showing 
significant  differences  varies  from  a  low  of  34%  (MOS  3 1C)  to  a 
high  of  93%  (MOS  IIB)  and  the  number  of  training-based  job  know¬ 
ledge  (K3)  subtests  showing  such  differences  varies  from  a  low  of 
33%  (MOS  5 IB)  to  a  high  of  100%  (MOS  IIB) . 


Additional  Analyses 

Because  of  the  sheer  volume  and  complexity  of  the  task-level 
data,  we  have  also  provided  several  summary  analyses  to  assist  in 
understanding  the  full  meaning  of  these  very  important  data. 

Interrelationships  Among  Performance  Measures 

The  first  of  our  additional  analyses  concerns  our  various 
measures  of  MOS  performance.  For  each  MOS,  the  scores  from  all 
of  our  performance  measures  are  moderately  to  highly  correlated. 
Because  of  these  relations,  more  reliable  measures  of  performance 
can  be  obtained  by  aggregating  scores  across  tasks  and  measures. 
Samples  of  the  results  of  these  analyses  are  shown  in  Table  2  of 
the  paper  by  Arabian  and  Mason,  provided  in  Appendix  C.  The  mean 
correlations  are  .68  for  training-based  knowledge  (K3)  with  job 
knowledge  (K5) ,  .41  for  training-based  knowledge  (K3)  with  hands- 
on  (HO),  and  .48  for  job  knowledge  (K5)  with  hands-on  (HO).  It 
should  also  be  noted  that  SQT  scores  are  moderately  correlated 
with  all  three  types  of  Project  A  performance  measures. 

Relationships  Between  AFOT  and  Core  Technical  Proficiency 

In  Appendix  E  are  tables  for  each  of  the  nine  Batch  A  MOS.  We 
completed  these  analyses  by  aggregating  the  performance  measures 
to  obtain  an  overall  MOS-specific  performance  score  (Core 
Technical  Proficiency)  for  each  soldier,  and  then  divided  the 
soldiers  into  three  groups;  those  scoring  in  the  upper,  middle, 
and  lower  thirds,  in  each  MOS,  on  the  overall  Core  Technical 
Proficiency  score.  The  specific  meaning  of  the  results  of 
partitioning  soldiers  into  thirds  obviously  is  dependent  on  the 
quality  of  soldiers  in  the  current  force.  That  is,  since  today's 
Army  has  many  high  quality  soldiers,  those  in  the  bottom  third 
today  are  in  reality  much  better  performers  than  we  would  have 
seen  several  years  ago.  Nonetheless,  we  have  found  these 
distinctions  useful  in  understanding  the  range  of  performance 
that  exists  even  among  high  quality  soldiers.  If  soldier  quality 
were  to  be  lowered,  we  would  expect  to  see  even  larger 
differences  in  performance. 

The  tables  show  the  relation  between  AFQT  category  and  overall 
MOS-specific  job  performance  (Core  Technical  Proficiency)  for 
each  of  these  MOS  separately.  They  also  indicate  the  number  of 
soldiers  in  each  AFQT  category  in  each  MOS  sample.  In  almost 
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every  case  the  percent  of  middle  and  high  performing  soldiers 
increases  with  increasing  AFQT.  We  would  place  little  importance 
on  those  few  cases  where  this  does  not  hold;  they  are  cases  where 
the  sample  size  is  too  small  to  be  reliable  (e.g.,  two  AFQT  IVB 
soldiers  in  the  MOS  13B  table) . 

Expected  Core  Technical  Proficiency  bv  AFQT 

Tables  illustrating  the  results  of  regression  analyses  for  the 
nine  Batch  A  MOS  are  provided  in  Appendix  F.  These  tables 
contain  data  on  the  expected  overall  Core  Technical  Proficiency 
for  soldiers  in  each  AFQT  category  having  10  or  more  soldiers. 

The  predicted  score  is  normalized  with  a  mean  of  100  and  a 
standard  deviation  of  20.  These  findings  show  that  across  MOS 
the  expected  MOS-based  technical  performance  of  soldiers 
increases  as  a  function  of  AFQT  category. 

Impact  Analyses 

The  next  set  of  analyses  addresses  the  question  of  the  impact 
of  observed  performance  differences  on  the  Army.  These  analyses 
are  based  on  the  partitioning  of  soldiers  into  thirds  based  on 
overall  Core  Technical  Proficiency  scores  as  described  above. 

What  does  it  mean  to  have  today's  soldier  in  the  top  third  or 
bottom  third  of  a  particular  performance  category  in  terms  of 
scores  on  particular  tasks?  To  help  answer  that  question,  we 
have  provided  a  summary  of  these  results  for  MOS  IIB  in  Appendix 
D.  This  summary  shows  how  soldiers  in  the  upper,  middle,  and 
lower  thirds,  overall,  performed  on  tasks/steps  of  the  various 
Project  A  performance  measures. 

Relationships  Between  art.f  and  Job  Performance 

The  final  set  of  analyses  we  present  here  concern  a  new  test 
developed  by  Project  A,  the  Assessment  of  Background  and  Life 
Experiences  (ABLE) .  This  test  was  developed  to  measure  attri¬ 
butes  that  are  related  to  the  motivational,  or  "will  do",  aspects 
of  job  performance  and  is  very  valid  for  this  purpose.  Histor¬ 
ically,  ASVAB  has  not  been  a  good  predictor  of  attrition  or  other 
"will  do"  performance  criteria.  This  is  not  surprising  since 
ASVAB  was  designed  to  predict  technical  MOS  performance,  which  it 
does  very  effectively,  and  not  the  motivational  components  of 
performance. 

ABLE  has  three  functional  scales  (1)  Achievement  Orientation, 

(2)  Dependability,  and  (3)  Physical  Condition.  Achievement 
Orientation  predicts  Effort  and  Leadership:  A  soldier's  ability 
to  exhibit  effort  across  the  entire  range  of  job  tasks,  persevere 
under  adverse  or  dangerous  conditions,  and  provide  peer  support. 
Dependability  predicts  Personal  Discipline:  The  degree  to  which 
a  soldier  exhibits  self-control,  adheres  to  Army  regulations  and 
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traditions,  and  avoids  disciplinary  actions.  The  Physical 
Condition  score  on  ABLE  predicts  Physical  Fitness  and  Military 
Bearing;  The  degree  to  which  a  soldier  maintains  good  physical 
conditioning  and  displays  proper  military  bearing. 

Data  on  the  relationship  between  scores  on  ABLE  and  the 
motivational  components  of  performance  are  shown  in  Appendix  H, 
pages  3-6.  Note,  for  example,  that  soldiers  scoring  in  the 
bottom  1/3  on  ABLE  have  almost  three  times  the  Article  15/flag 
action  rate  of  those  in  the  top  1/3,  while  ASVAB  does  not 
differentiate  between  those  with  high  and  low  disciplinary  rates. 

Data  on  the  unique  and  incremental  validities  of  ABLE  are 
presented  in  Appendix  H,  page  7.  When  we  examine  ABLE  as 
compared  to  education  and  ASVAB,  we  find  that  ABLE  is  the  single 
strongest  predictor  of  the  three  "will  do"  performance  factors. 

It  adds  much  more  predictive  power  to  ASVAB  than  education  does. 

Summary 

From  the  data  presented  in  this  paper  four  conclusions  emerge: 
(1)  AFQT  is  closely  related  to  technical  proficiency,  although 
this  relationship  varies  with  type  of  performance  measure,  as 
well  as  by  task,  (2)  reasonable  interrelationships  have  been 
found  among  Project  A  job  performance  measures;  (3)  the 
performance  measures  do  a  good  job  of  differentiating  performance 
among  soldiers;  and  (4)  a  new  Project  A  measure,  ABLE,  predicts 
the  motivational  aspects  of  job  performance  that  the  ASVAB  was 
never  intended  to  predict. 

Information  from  the  Project  A  concurrent  validation  data 
collection  in  1985  discussed  here,  and  from  the  follow-on  major 
longitudinal  validation  data  collection,  scheduled  for  summer  of 
FY88,  will  continue  to  provide  a  unique  data  set,  unparalleled  in 
richness  for  addressing  recurring  Army  concerns.  These  data  are 
invaluable  to  the  Army  in  the  areas  of  accession  policy,  setting 
standards  for  enlistment  and  reenlistment,  predicting  attrition 
and  reenlistment,  linking  school  training  to  field  performance, 
and  assisting  in  the  accomplishment  MANPRINT  requirements. 
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Introduction 


Beginning  in  1979  military  testing  has  been  the  focus  of 
increasing  attention,  from  both  the  Department  of  Defense  and 
Congress.  The  reasons  for  this  interest  are  varied  and  include: 

1)  The  discovery  in  1979  that  the  then  operational  selection 
test  battery  for  military  enlistment  was  misnormed. 

2)  The  switch  in  reference  groups  for  military  tests  from 
the  1944  mobilization  population  (all  men,  including 
officers,  serving  under  arms  during  that  year)  to  the  1980 
youth  population  (aJ.l  17-  to  23-year  olds  in  1980 
including  both  males  and  females) . 

3)  A  Congressional  mandate  to  link  entry  test  scores  directly 
to  on-the-job  performance  rather  than  to  traditional 
training  performance. 

4)  Intense  interest  in  combining  computer  technology  with 
emerging  developments  in  testing  theory. 

As  an  integrated  response  to  the  various  concerns,  the  U.S. 
Army  Research  Institute  designed  Project  A,  "Improving  the 
Selection,  Classification,  and  Utilization  of  Army  Enlisted 
Personnel,"  a  multi-year,  multi-million  dollar  research  effort. 
Project  A  had  many  objectives  including  (1)  continued  validation 
of  the  operational  Armed  Forces  Vocational  Aptitude  Battery 
(ASVAB) ,  (2)  development  and  validation  of  new  cognitive  and  non- 

cognitive  measures  that  might  expand  the  coverage  of  the 
predictor  domain  beyond  that  of  the  ASVAB  in  predicting  total 
soldier  performance,  and  (3)  development  of  a  comprehensive  set 
of  performance  criteria  including  paper-and-pencil  tests  of 
school  knowledge  and  job  knowledge,  hands-on  tests  of  job 
performance,  and  behaviorally-anchored  rating  scales. 

Project  A  was  initiated  in  1982  and  will  be  completed  in 
1989.  While  Project  A  focused  primarily  on  Army  enlisted 
personnel  in  their  first  tour  of  service,  a  second  research 
program,  "Building  and  Retaining  the  Career  Force,"  is  scheduled 
to  begin  in  1990.  This  effort  will  focus  on  the  Army's  career 
force,  those  soldiers  who  reenlist  and  remain  beyond  one  tour  of 
duty.  Soldiers  who  comprise  the  career  force  provide  not  only 
leadership  and  continuity  to  the  Army  as  an  institution,  but  are 
crucial  in  insuring  that  the  Army  meets  its  battlefield  mission. 

Another  accomplishment  by  the  Army  in  military  testing  during 
the  last  decade  has  been  the  development  and  implementation  of 
the  Computer  Adaptive  Screening  Test  (CAST) .  CAST  is  used  in 
recruiting  stations  to  determine  whether  a  prospect  is  likely  to 
score  well  enough  on  the  ASVAB  to  be  considered  for  enlistment. 
Prior  to  implementation  of  CAST  all  prospects'  eligibility  to 
enlist  had  to  be  determined  by  transporting  them  to  a  testing 


site  to  take  the  three-and-a-half  hour  ASVAB.  The  use  of  CAST 
has  reduced  the  number  of  ASVAB  tests  given  to  Army  prospects 
from  over  one  million  per  year  in  the  early  1980s  to  approximate¬ 
ly  400,000  per  year  currently,  while  the  number  of  accessions  per 
year  has  remained  about  the  same. 

Recognizing  that  large  scale  efforts  like  Project  A  will  not 
be  feasible  for  all  Army  jobs,  an  exploratory  research  effort, 
"The  Army  Synthetic  Validation  Project,"  is  currently  underway. 
This  program  is  designed  to  (1)  develop  procedures  for  deriving 
prediction  equations  for  MOS  using  primarily  the  extensive  data 
gathered  on  the  relatively  small  sample  of  MOS  in  Project  A,  and 
(2)  develop  procedures  for  setting  selection  standards  linked  to 
job  performance. 

The  remainder  of  this  paper  will  more  fully  describe  these 
efforts  as  well  as  some  of  the  resulting  products  that  have  been 
implemented  either  operationally  or  in  testbeds.  Future  plans  in 
the  area  of  military  testing  will  also  be  discussed. 

Project  A 

In  Project  A,  the  research  objectives  are  to: 

1)  Develop  new,  comprehensive  measures  to  cover  the  job 
performance  domain  in  the  Army.  These  measures  include 
both  Army-wide  and  MOS-specific  rating  scales,  written 
school  and  job  knowledge  tests,  and  direct  hands-on 
measures  of  MOS-specific  task  proficiency. 

2)  Validate  ASVAB  against  both  existing  and  new  performance 
measures . 

3)  Develop  and  validate  new  selection  and  classification 
measures  against  existing  and  new  performance  measures. 

4)  Validate  intermediate  criteria  (such  as  training  perfor¬ 
mance)  ,  as  predictors  of  later  criteria,  such  as 
first-tour  job  performance,  so  that  better  informed 
decisions  on  reenlistment  can  be  made. 

5)  Examine  the  validity  and  utility  of  alternative 
procedures  for  making  operational  selection  and  clas¬ 
sification  decisions  in  the  Army. 

The  Project  A  research  design  consists  of  three  main  stages 
of  data  collection  and  analysis  in  an  iterative  progression  of 
development,  testing,  evaluation,  and  further  refinement  of  the 
selection/classification  instruments  (predictors)  and  measures 
of  job  performance  (criteria) .  In  the  first  stage,  individual- 
level  data  from  FY  81-82  were  examined  to  explore  relationships 
between  soldiers'  ASVAB  scores  and  their  later  performance  in 
training  and  on  first-tour  Skill  Qualification  Tests  (SQT) . 


Prior  to  the  second  stage  of  the  data  collection,  19  Military 
Occupational  Specialties  (MOS)  were  selected  as  a  representative 
sample  of  the  Army's  250+  entry-level  jobs.  These  MOS  were 
selected  based-x>n~“lr)— ^-^lustering  of  MOS  based  on  similarity  of 
job  conter(€  job  content^  2)  the  fact  that  these  MOS  are 
representa^ve-of-^AHD^f-^bs  in  general  and  account  for  Over  45% 
of  the  Army's  accessions,  and  3)  the  recommendations  of  General 
Officers  representing  major  Army  Commands  at  that  time.  Nine  of 
the  19  MOS  were  then  selected  to  have  both  paper-and-pencil  and 
hand-on  tests  of  job  performance  in  addition  to  the  school 
knowledge  tests  and  ratings  administered  in  all  of  the  MOS.  The 
nine  fully-tested  MOS  in  Project  A  are:  Infantryman  (IIB) ,  Cannon 
Crewman  (13B) ,  Tank  Crewman  (19E) ,  Radio  Operator  (31C) ,  Light 
Wheel  Vehicle  Mechanic  (63B) ,  Administrative  Specialist  (71L) , 
Motor  Transport  Operator  (88M),  Medical  Care  Specialist  (91A) , 
and  Military  Police  (95B) . 

The  second  stage  of  the  Project  A  research,  the  Concurrent 
Validation  (CV) ,  was  conducted  in  FY  1985.  During  the  CV,  over 
9400  soldiers  in  the  19  MOS  were  administered  the  new  Project  A 
predictor  tests,  which  included  measures  of  spatial  abilities, 
temperament  and  vocational  interest,  as  well  as  computerized 
tests  of  perceptual  and  psychomotor  skills.  These  tests  were 
designed  to  expand  the  predictor  domain  in  teras  of  individual 
characteristics  and  attributes  that  might  be  important  for 
selection  in  predicting  more  aspects  of  performance.  The  new 
predictors  were  intended  to  supplement  those  cognitive  abilities 
already  assessed  by  ASVAB. 

Concurrently,  the  CV  soldiers,  who  had  been  in  the  Army  18 
to  27  months,  were  administered  a  comprehensive  set  of  job 
performance  measures  including  supervisory  and  peer  ratings, writ¬ 
ten  school  and  job  knowledge  tests,  and  MOS-specific  hands-on 
task  proficiency  measures. 

One  of  the  major  scientific  contributions  of  the  Project  A 
research  to  date  is  a  comprehensive  modeling  of  the  job  perfor¬ 
mance  domain.  The  criterion  development  efforts  in  the  project 
v/ere  driven  by  the  idea  that  job  performance  is  multi— dimensional 
and  that  performance  is  best  measured  through  a  variety  of 
methods.  Analyses  of  the  criterion  data  collected  during  the  CV 
resulted  in  five  empirically  derived  dimensions  that  represent 
overall  Army  job  perf oirmance .  The  five  Project  A  job  performance 
dimensions  are: 

1)  MQS-Technical  Knowledge  and  Skill.  The  proficiency  with 
which  a  soldier  performs  the  technical  tasks  that  are 
critical  and  central  to  his/her  MOS. 

2)  General  Soldiering  Proficiency:  How  well  a  soldier 
executes  common  soldiering  tasks  such  as  first  aid  and 
land  navigation. 

3)  Effort  and  Leadership:  The  degree  to  which  a  soldier 


exerts  effort  over  the  full  range  of  job  tasks,  perse¬ 
veres  under  adverse  or  dangerous  conditions,  and  demon¬ 
strates  leadership  and  support  toward  peers. 

4)  Personal  Discipline:  The  degree  to  which  a  soldier 
adheres  to  Army  regulations,  exhibits  self-control  and 
does  not  create  disciplinary  problems. 

5)  Physical  Fitness  and  Military  Bearing:  The  degree  to 
which  a  soldier  maintains  an  appropriate  military 
appearance  and  remains  in  good  physical  condition. 

The  first  two  performance  dimensions  involve  a  soldier's 
ability  to  perform  the  technical  requirements  of  his/her  job  and 
are  often  referred  to  as  "can  do"  factors.  The  remaining  three 
performance  dimensions  are  more  attitudinal  or  motivational  in 
nature  and  are  often  called  "will  do"  factors. 

The  third  stage  of  the  Project  A  research,  known  as  the 
Longitudinal  Validation  (LV) ,  started  in  1986  with  the  admin¬ 
istration  of  the  battery  of  new  Project  A  predictor  tests  to 
approximately  55,000  new  recruits  in  21  MOS.  At  the  end  of 
his/her  Advanced  Individual  Training  (AIT)  or  One  Station  Unit 
Training  (OSUT) ,  each  soldier  was  administered  the  school 
knowledge  test  for  his/her  MOS,  and  a  set  of  rating  scales  was 
collected  from  supervisors  and  peers.  Approximately  11,000  of 
these  soldiers  were  followed  into  their  first-tour  field 
assignment  and  have  been  administered  a  set  of  MOS— specific  and 
Army-wide  job  performance  measures  during  FY  88  and  FY  89.  In 
addition,  about  1,000  of  the  soldiers  from  the  CV  sample  who 
reenlisted  have  been  assessed  on  a  battery  of  performance 
measures  appropriate  for  second-tour  including  rating  scales, 
written  job  knowledge  tests,  hands-on  measures,  and  leadership 
measures. 

Results  and  Products 

During  the  execution  of  the  research  plan.  Project  A  made 
many  scientific  contributions  to  the  field  of  Industrial  and 
Organizational  psychology  and  so  far  has  delivered  four  major 
products  to  the  Army. 

ASVAB.  One  product  which  resulted  from  the  Concurrent 
Validation  involves  improvements  in  the  computation  and  use  of 
ASVAB  Aptitude  Area  (AA)  Composites.  These  composites  are  used 
to  determine  eligibility  for  training  in  Army  jobs.  Beginning 
with  the  operational  implementation  of  ASVAB  11/12/13/14  in 
October  1984  the  formulae  for  the  Clerical  (CL)  and  the  Surveil¬ 
lance/Communications  (SC)  composites  were  changed.  These  changes 
resulted  in  both  better  accuracy  and  improved  fairness  in  the 
prediction  of  job  performance  for  minorities  in  several  MOS.  In 
addition,  a  change  in  the  computation  of  the  Mechanical  Main¬ 
tenance  (MM)  composite  and  recommended  changes  in  required 
aptitude  areas  for  approximately  50  MOS  are  scheduled  for 


implementation  in  the  near  future.  Annual  savings  from  the 
changes  in  composites  have  been  estimated  at  $25  million. 

The  Enlisted  Personnel  Allocation  System  (EPAS)  is  a  new 
assignment  system  that  will  more  efficiently  match  qualified  Army 
applicants  to  jobs  for  which  they  are  best  qualified,  maximizing 
performance  and  minimizing  attrition.  Thus,  with  significant 
improvements  in  classification  composites  (from  Project  A)  and  in 
assignment  (from  EPAS)  the  Army  can  put  the  "right"  person  in  the 
"right"  job  at  the  "right"  time.  The  potential  savings  to  the 
Army  from  using  ASVAB  (including  the  improved  composites)  and  the 
Enlisted  Personnel  Allocation  System  to  fill  accession  rec[uire- 
ments  is  estimated  to  be  more  than  $480  million  annually. 

Results  from  the  Project  A  CV  have  demonstrated  conclu¬ 
sively  that  ASVAB  is  an  excellent  test  of  the  "can  do"  or  more 
technical  task-based  requirements  of  Army  jobs.  The  mean  validity 
coefficient  between  scores  on  ASVAB  and  Core  Technical  perfor¬ 
mance  is  .57  across  the  19  MOS  in  the  Concurrent  Validation. 

Since  ASVAB  was  designed  as  a  general  cognitive  test,  it  is  not 
expected  to  be  a  good  measure  of  more  specialized  spatial 
abilities,  psychomotor  skills,  motivation,  interests  or  leader¬ 
ship.  As  a  means  of  supplementing  ASVAB,  the  new  Project  A  tests 
measure  these  characteristics  very  well. 

Spatial.  Psvchomotor  and  Perceptual  Tests.  Currently,  ARI 
is  supporting  implementation  of  Project  A's  spatial  and  com¬ 
puterized  psychomotor  and  perceptual  tests  in  USAREUR  and  at 
training  posts  in  CONUS. 

These  implementations  were  inspired  by  ARI  selection 
research  on  tank  and  anti-tank  gunners.  In  1986,  Project  A's  new 
tests  had  a  multiple  R  of  .76  against  accuracy  on  high-tech 
simulators  of  tank  gunnery  for  95  Armor  officers  at  Fort  Knox. 
Cognitive  ability,  which  was  tested  as  well,  did  not  contribute 
to  the  prediction.  In  1987,  300+  new  recruits  at  Fort  Banning 
took  the  same  battery  before  training  in  anti-tank  gunnery. 
Several  of  the  new  tests  strongly  predicted  accuracy  in  firing 
the  TOW  (Tube-launched,  Optically-tracked,  Wire-guided)  missile 
simulator. 

Based  on  these  results,  in  December  87,  CG  TRADOC  ordered 
implementation  of  Project  A  tests  at  four  posts  that  train  on 
high-tech  weapons  systems.  ARI  installed  1-  and  2-hand  tracking, 
maze,  and  mental  rotation  tests  at  Forts  Knox,  Penning,  and  Bliss 
in  February  88. 

At  the  Infantry  and  Armor  sites,  the  earlier  results  have 
been  confirmed.  For  1,065  TOW  students  to  date,  41%  of  those 
passing  a  cut  score  on  the  predictors  qualified  as  gunners  in  the 
minimum  time,  as  opposed  to  24%  of  those  not  tested  on  the 
psychomotor/spatial  predictors  and  12%  of  those  scoring  below 
the  cut  score.  Those  scoring  above  the  cut  score  qualified 
higher  as  well  as  faster:  48%  attained  the  upper  two  levels  of 


accuracy  as  contrasted  with  36%  of  those  not  tested  and  31%  of 
those  below  the  cut  score.  In  terms  of  validities,  accuracy  was 
predicted  at  .37  by  the  Project  A  tests,  .29  by  ASVAB  GT,  and 
.38  by  the  two  together.  In  mid-88,  these  training  data  were 
confirmed  in  live  fire  at  Fort  Banning:  for  60  students  firing 
one  live  TOW  each  at  a  moving  target  at  6,000',  P(hit)  was  .85 
for  those  who  met  the  cut  on  the  Project  A  tests  and  .73  for 
those  who  did  not. 

At  Fort  Knox,  Armor  recruits  (N  «  500+)  scoring  in  the 
upper  third  on  the  new  predictors  early  in  training  later  had 
gunnery  hit  rates  16%  higher  than  those  in  the  lower  third, 
exactly  repeating  the  results  in  the  1986  research.  The  new 
predictors  correlated  .54  with  speed/accuracy,  compared  with  a 
.34  validity  for  the  ASVAB  GT  composite.  Combined,  the  validity 
was  .55,  reconfirming  the  importance  of  spatial  and  psychomotor 
skills'  in  predicting  gunnery  performance.  At  both  posts,  the 
new  results  were  so  positive  that  the  test  scores  were  incor” 
porated  into  the  decision-making  process. 

Validities  at  Fort  Bliss  were  not  significant,  and  testing 
there  has  been  suspended.  In  contrast  to  tank  and  anti-tank 
weapons,  which  rely  heavily  on  tracking,  the  air  defense  systems 
tested  use  fire-and-forget  weapons  against  fleeting  targets. 

Thus  these  systems  rely  more  on  skills  like  vigilance  and  target 
identification. 

Completing  the  implementation  requests  from  CG  TRADOC,  the 
Field  Artillery  School,  Fort  Sill,  is  starting  to  administer  a 
broader  battery  of  the  new  tests  to  rec^its  in  meteorology, 
surveying,  radar  range- finding,  and  artillery  spotting  (MOS) . 
Validation  will  be  against  performance  at  the  end  of  training. 

Currently,  the  active  forces  are  transitioning  to  the 
Bradley  Fighting  Vehicle.  In  USAREUR,  V  Corps  has  elected  to 
use  a  battery  of  spatial  and  psychomotor  tests  to  inform  their 
selection  of  Bradley  gunners  in  the  3rd  Armor  Division  and  the 
8th  Infantry  Division. 

A  need  for  improved  selection  has  been  found  also  at  the 
Special  Forces  School,  where  attrition,  due  primarily  to 
failures  in  land  navigation,  is  costly.  In  a  new  pilot  project, 
ARI  has  installed  three  spatial  tests  at  Fort  Bragg  to  identify 
good  land  navigators. 

In  review,  early  positive  results  are  being  replicated  in 
the  implementations;  spatial/psychomotor  abilities  strongly 
predict  differences  in  gunneiry  performance;  and  utilization  is 
spreading.  Budget  reductions,  however,  threaten  the  survival  of 
the  implementation  in  the  training  base.  In  the  future,  the 
tests  could  have  the  greatest  impact  if  they  were  administered 
before  enlistment.  In  that  case  the  Army  Reserve  and  National 
Guard  could  be  served  as  well,  and  initial  person— 30b  matching 
could  be  strengthened  in  many  MOS  by  adding  spatial  and/or 


psychomotor  abilities  to  the  profile  of  aptitudes. 

Assessment  of  Background  and  Life  Experiences  fABLE) .  The 
ABLE  is  a  30-minute,  multiple  choice,  non-cognitive  test  designed 
to  measure  temperament,  personal  history,  and  adaptability.^  In 
Project  A,  ABLE  was  shown  to  improve  significantly  the  prediction 
of  the  motivational  components  of  performance.  Scores  on  the 
Adjustment  scale  of  ABLE  are  strongly  related  to  12-month 
attrition.  Recruits  who  have  very  low  Adjustment  scores  have 
attrition  rates  that  are  two  to  three  times  higher  than  soldiers 
with  high  Adjustment  scores. 

The  Dependability  scale  of  ABLE  predicts  in-service  discipli¬ 
nary  problems.  For  example,  soldiers  having  low  Depend2±>ility 
scores  receive  significantly  more  Articles  15  than  those  with 
high  scores.  Conversely  soldiers  with  high  Dependability  scores 
are  more  often  viewed  as  having  potential  of  becoming  high 
performing  NCOs. 

In  sum,  the  ABLE  shows  promise  to  augment  the  Services' 
capability  to  identify  the  most  qualified  applicants  in  all  AFQT 
categories  and  educational  levels  by  predicting  the  "will  do" 
components  of  performance.  Based  on  these  encouraging  results, 
the  ABLE  is  included  in  a  joint  Se^ice  adaptability  screening 
instrument  that  will  become  operational  in  1990. 

The  Longitudinal  Research  Data  Base.  The  fourth  and  perhaps 
most  enduring  product  from  the  Project  A  research  effort  is  the 
Longitudinal  Research  Data  Base  (LRDB) .  The  LRDB  is  a  permanent 
storehouse  of  empirical  information  on  Project  A,  unparalleled  in 
its  richness  for  addressing  recurring  Army  concerns.  These  data 
are  invaluable  to  the  Army  in  the  areas  of  accession  policy, 
setting  standards  for  enlistment  and  reenlistment,  predicting 
attrition,  linking  school  training  to  field  performance,  and 
linking  characteristics  at  entry  into  the  military  to  performance 
in  first  and  second  tours  of  duty. 

Building  and  Retaining  the  Career  Force 

At  the  end  of  the  Project  A  Longitudinal  Validation  data 
collections  in  FY  89,  the  first  phase  of  the  Army's  research 
program  to  build  a  new  selection  and  classification  system  , based 
on  job  performance,  will  be  completed.  The  second  phase,  which 
will  include  measures  of  critical  tasks  for  soldiers  in  their 
second  tour,  will  be  initiated  in  FY  90. 

The  research  objectives  planned  for  Building  and  Retaining 
the  Career  Force  are  to; 

1)  Develop  a  set  of  measures  for  selecting  and  classifying 
enlisted  personnel  in  order  to  optimize  second  tour 
soldier  performance  without  sacrificing  first  tour 
performance. 


2)  Analyze  longitudinal  data  from  Project  A  and  other 
sources  to  develop  information,  procedures,  and  recommen¬ 
dations  for  implementing  new  selection  and  classification 
measures  in  building  the  Army's  career  force. 

3)  Determine  the  validity  of  ASVAB,  new  predictors, 
training  performance,  and  first  tour  performance  in 
predicting  future  performance  (including  second  tour) . 

Results  generated  from  both  Project  A  and  Building  and 
Retaining  the  Career  Force  will  be  useful  to  Army  policy  makers 
in  setting  enlistment  standards  and  to  Army  training  proponents 
in  setting  quality  distribution  goals  during  years  of  a  declining 
manpower  pool .  The  outcomes  will  also  be  helpful  to  DoD  in  ^ 
providing  (1)  more  tools  for  conducting  selection  and  classifica¬ 
tion  testing,  and  (2)  scientific  data  to  use  in  responding  to 
Congress  and  the  general  public  on  questions  concerning  military 
testing. 


Computerized  Adaptive  Screening  Test  (CAST) 

The  Computerized  Adaptive  Screening  Test  (CAST)  is  a  tool 
for  recruiters  to  estimate  the  likely  performance  of  a  prospect 
on  the  Armed  Forces  Qualification  Test  (AFQT)  of  the  ASVAB.  As 
the  name  suggests,  CAST  is  a  computer  adaptive  test,  and  it  is 
administered  in  recruiting  stations.  The  computer  selects 
succeeding  items  based  on  responses  to  previous  items.  Because 
item  difficulty  is  matched  to  examinee  ability,  maximum  informa¬ 
tion  is  obtained  from  each  item.  Thus,  a  short  CAST  can  provide 
as  much  information  as  a  longer  paper-and-pencil  test. 

CAST  was  preceded  by  the  Enlistment  Screening  Test  (EST) . 

The  EST  was  a  good  predictor  of  individuals'  scores  on  the  AFQT, 
but  it  had  two  problems.  First,  recruiters  felt  it  was  too  long, 
and  second,  the  EST  had  to  be  hand  scored.  The  implementation  of 
JOIN  (Joint  Optical  Information  Network)  provided  computers  for 
recruiting  stations,  and  computer-administered  testing  became 
feasible.  Whereas  the  EST,  a  paper-and-pencil  test,  takes  45 
minutes,  CAST  can  be  completed  in  15  minutes. 

CAST  was  initially  developed  by  the  Naval  Personnel  Research 
and  Development  Center  (NPRDC)  in  a  cooperative  arrangement  with 
ARI.  A  field  test  on  364  Army  applicants  showed  a  validity  of 
.85  using  AFQT  as  the  criterion.  This  was  an  encouraging 
finding,  comparable  to  the  validity  of  .83  for  EST.  Two  later 
cross-validations  on  larger  samples  yielded  comparable  results 
(i.e.  validities  of  .80  and  .79). 

The  initial  implementation  of  CAST  received  a  generally 
positive  reception  from  recruiters  and  was  successful  in 
predicting  operational  AFQT.  However,  some  improvements  were 
necessary,  leading  to  recent  refinements  in  CAST.  The  number  of 
items  in  CAST  was  increased  from  approximately  300  to  over  500. 
This  increase  was  needed  to  reduce  the  likelihood  that  any 


particular  item  would  be  compromised  as  a  result  of  overuse.  In 
addition,  a  better  format  for  reporting  results  was  needed.  The 
original  display  predicted  a  specific  AFQT  score.  What  the 
recruiter  and  prospect  really  wanted  to  know,  however,  was  the 
likelihood  that  the  prospect  would  score  in  certain  critical 
ranges  of  the  AFQT.  Accordingly,  the  final  display  was  rede¬ 
signed  to  meet  that  need.  The  current  version  of  CAST  also  has 
been  calibrated  to  the  new  AFQT  introduced  operationally  in 
January  1989. 


Synthetic  Validation  Project 

The  Army's  Synthetic  Validation  (SynVal)  Project  is  an 
exploratory  research  effort  currently  underway.  The  concept  of 
synthetic  validity  was  first  introduced  by  Lawshe  (1952)  as  a 
middle  ground  between  situational  validity,  which  requires 
costly  validity  analyses  (empirical  validation)  for  each  job,  and 
generalized  validity,  which  assumes  the  validity  of  a  test  across 
similar  jobs.  The  Project  A  research,  with  its  comprehensive  set 
of  new  predictor  tests  (non-cognitive,  spatial,  and  psychomotor) 
in  addition  to  the  existing  ASVAB  and  extensive  job  performance 
measures,  afforded  a  unique  opportunity  to  examine  SynVal  as  a 
cost-effective  alternative  to  empirical  validity  studies  for  a 
large  number  of  MOS. 

This  project  has  two  main  goals: 

1)  Develop  procedures  for  identifying  job  performance 
prediction  equations  for  new  MOS,  low-fill  MOS,  and  for 
other  MOS  when  it  is  impractical  or  too  costly  to  derive 
empirical  prediction  equations. 

2)  Develop  procedures  for  establishing  cut  scores  on 
predictor  tests  that  are  linked  to  job  performance  and 
that  identify  both  minimally  qualified  and  highly 
qualified  recruits. 

The  research  design  has  three  phases,  each  with  a  data 
collection.  The  initial  phase  concentrates  on  MOS  in  the  Project 
A  sample  that  had  the  full  complement  of  criterion  measures; 
the  second  phase  adds  MOS  which  had  reduced  criterion  testing; 
and  the  final  phase  will  apply  the  SynVal  procedures  to  at  least 
one  MOS  that  was  not  included  in  Project  A.  Comparisons  will  be 
made  among  job  component  models  and  among  standard  setting 
procedures. 

Three  job  component  models  have  been  developed  and  tested; 
(1)  Job  Behaviors  Model,  (2)  Attribute  Model,  and  (3)  Job  Tasks 
Model.  The  results  of  phase  one  have  been  extremely  encouraging. 
Army  subject  matter  experts  (SMEs)  were  able  to  reliably  use  the 
three  models  to  identify  job  components  (in  terms  of  importance 
or  estimated  validity)  for  each  of  the  three  MOS.  The  resulting 
job  description  (component)  profiles  differed  systematically 
across  MOS.  Furthermore,  the  average  "synthesized"  validity 


(across  ^he  l^hres  phase  one  jobs  and  models)  compared  favorably 
with  the  average  empirical  validity  (.50  vs  .67,  respectively). 
However,  given  the  exploratory  nature  of  these  findings,  it  would 
be  premature  to  exclude  any  model  or  weighting  scheme. 

with  respect  to  the  second  goal,  an  extensive  literature 
review  found  little  research  on  procedures  for  setting  cut  scores 
that  are  linked  to  job  performance.  The  project  assumes  that 
requirements  for  different  levels  of  job  performance  must  be 
determined  first.  Then  cut  scores  could  be  set  on  the  predictor 
(selection)  measure (s)  to  optimize  the  likelihood  of  obtaining 
individuals  who  would  meet  the  specific  performance  requirements. 

Four  performance  levels  have  been  objectively  defined: 
Unacceptable,  Marginal,  Acceptable,  and  Outstanding.  The  three 
standard  setting  approaches  developed  and  tested  in  phase  one 
were:  (1)  Soldier-based,  (2)  Task-based,  and  (3)  Critical 

Incident-Based.  Although  mixed  results  have  been  obtained,  the 
existing  literature  on  standard  setting  also  consistently 
indicates  differences  in  obtained  standards  with  different 
techniques.  The  critical-incident  approach  resulted  in  the  most 
lenient  standards  for  minimal  performance  followed  by  the 
soldier-based  approach  with  the  task-based  approach  being  the 
most  stringent.  However,  the  task-based  approach  resulted  in 
more  variation  (less  agreement)  among  the  judges  while  the 
critical-incident  approach  showed  the  greatest  agreement. 

A  third  of  the  way  through  this  project,  the  results  are 
encouraging.  In  making  use  of  the  Project  A  LRDB  to  extend 
validity  information  from  a  relatively  small  sample  to  potential¬ 
ly  all  entry-level  MOS,  SynVal  represents  a  major  return  on  the 
Project  A  investment.  Future  research,  building  on  this  foun¬ 
dation,  might  include  testing  the  transferability  of  the  SynVal 
methodologies  to  the  other  Services  as  well  as  to  the  weapon 
system  acquisition  process. 
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ASVAB  RETEST  SCORE  GAINS 


Karen  J.  Mitchell 
Lawrence  M.  Hanser 


INTRODUCTION 

The  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  was 
introduced  in  January  1976  as  the  single  Department  of  Defense 
selection  and  classification  battery.  It  is  administered  to  over 
one  million  applicants  for  military  service  each  year.  In  FY83 
over  450,00.0  nonp’rior-service  Army  applicants  were  tested  on 
ASVAB,  forms  8,  9  and  10  i 

The  ASVAB  is  composed  of  eight  power  and  two  speed  tests.  Four 
ASVAB  subtests  are  combined  to  form  the  Armed  Forces 
Qualification  Test  (AFQT) ,  a  general  measure  of  traihabillty  and 
the  primary  criterion  for  enlistment.  The  AFQT  cutoff  for  FY83 
Army  accession  was  set  at  the  16th  percentile  for  high  school 
graduates  and  the  31st  percentile  for  nongraduates.  In  FY83  85.1 
percent  of  the  Army  applicants  met  AFQT  standards  on  initial 
testing  while  13.4  percent  failed  to  meet  the  AFQT  enlistment 
criterion  on  the  initial  score-o"f -record .  The  remaining  1.4 
percent  were  selected  for  verification  retesting.  Examinees  are 
‘^■asked  to  verify  test  scores  by  reexamining  on  the  AFQT  when 
inconsistencies  in  performance  are  noted  on  various  parts  of  the 
test. 

During  FY83  11.8  percent  of  Army  applicants  took  the  ASVAB  two  or 
more  times.  Examinees  may  retest  because  (1)  they  fail  to  meet 
the  enlistment  cutoff  on  initial  testing,  (2)  they  wish  to ' 
Increase  their  scores  on  the  aptitude  area  composites  which  are 
used  for  occupational  classification,  or  (3)  they  are  asked  to 
repeat  the  test  for  verification  purposes.  Of  the  approximately 
53,000  Army  applicants  who  retested  in  FY83,  26.6  percent  had 
failed  on  initial  testing;  62.1  percent  had  passed,  and  11.4 
percent  were  verification  retesters. 

Sims  and  Maier  (1983)  and  Friedman  (1983)  have  examined  test 
score  gains  for  examinees  taking  ASVAB  two  or  more  times.  Their 
work  shows  that  applicants  show  consistent  and  meaningful  score 
gains  on  retesting.  It  is  hypothesized  that  examinees  become 
familiar  with  test  booklet  formats,  answer  sheet  layouts,  test 
content,  pacing  requirements,  etc.  on  multiple  testings.  The 
examinee,  thereby,  acquires  test-taking  skills  and  content 
knowledge  that  facilitate  performance  on  subsequent  testings. 
These  test  score  gains  are  expected  to  be  approximately 
equivalent  across  the  subtests  of  ASVAB. 


'sims  and  Maier  (1983)  and  Friedman  (1983),  however,  have  shown 
that  retest  gains  for  the  two  ASVAB  speed  subtests,  Numerical 
Operations  (NO)  and  Coding  Speed  (CS) ,  exceed  those  for  the  ASVAB 
power  subtests  and  that  the  speeded  test  gain  (in  sigma  units) 
is  roughly  twice  the  size  of  the  gain  observed  for  the  other 
tests.  The  purpose  of  this  research  is  to  examine  the  impact  of 
these  gains  on  enlistment  qualification  in  the  Army. 


METHOD 
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RESULTS  AND  DISCUSSION 

Mean  AFQT  scores  computed  at  initial  testing  and  retest  using  the 
operational  procedure  appear  at  Table  1  for  retesters  who  (U 
failed  to  meet  accession  standards,  (2)  met  the  AFQT  criterion, 
and  (3)  were  asked  to  verification  test  at  Initial  testing. 

^Mean  AFQT  gains  from  initial  to  retest  also  appear  for  the  three 

groups. 


TABLE  1 

MEAN  AFQT  INITIAL  AND  RETEST  PERCENTILE  SCORES 


Ini 

Failed 

15 

Passed 

31 

Verified 

39 

tial 

Retest 

.81 

22.89 

.72 

37  .97 

.18 

45.76 

Mean  Difference 

7.07 

6.25 

6.58 


Of  the  retesters  who  failed  to  meet 
testing,  45.6  percent  qualified  on 
applicants  43  percent  enlisted. 


the  AFQT  cutoff  on  Initial 
their  retests.  Of  these 


Table  2  presents  AFQT  data  for  initial  testing  and  for 
testing  with  AFQT  computed  using  the  retest  AR,  WK,  and 
and  initial  NO  score.  Mean  AFQT  scores  and  mean  gains 
initial  to  second  testing  are  displayed. 


a  second 
PC  scores 
from 
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TABLE  2 

MEAN  AFQT  INITIAL  AND  RECOMPUTED  RETEST  PERCENTILE  SCORES 

Initial  Retest  Mean  Difference 

Failed  15.81  20.22  4.A1 

Passed  31.72  35.61  3.90 

Verified  39.18  42.69  3.51 

The  mean  retest  gain  reported  in  Table  2  is  from  2.3  to  3.1 
points  lower  than  that  presented  in  Table  1.  Of  the  retesters 
who  had  failed  to  meet  enlistment  standards  initially,  35.6 
percent  qualified  for  enlistment  when  AFQT  was  computed  in  this 
way.  Operationally,  this  scoring  procedure  may  be  implemented  by 
maintaining  Initial  NO  and  CS  scores  in  an  applicant's  retest 
record  and  using  the  first-time  speed  scores  and  retest  power 
scores  in  computation  of  AFQT  and  the  occupational  composites. 
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..d  wtltude  area  compoalte.  (Hal.t  i  Sima.  1982;  tea.  Valantlaa  4  Earlas. 
1983) . 

r  LnToKr'4:s 

sheets  used  for  military  entrance,  testing. 

.  *  loa^  wariAs  Guiliano.  Bee  &  Valentine  outlined  a  plan 

aa^uS  Ta^/,  ^“aductad 

coraecaloaa  for  tte  N08C  apa^ 

percentile  conversion  tables.  ®  -  .qaA  Thev  will  be 

intr^actloa  of  AS^M  foraa  11/12/13. 

PROBLEM 

Ofara  1944  varloua  forma  of  AFQT  and  ASV4S  hava  bean  callbtatad  to  tha 
»orld  war  II  mobilization  P«P»1a«-J  “tte°Uuror,ppu'cira'and 

claaalflcatlon  atandarda  acroaa  Mat  ^  “naarch'la  naadad  to 

rnmirib:  rtSa°" 

r^trat"  r 0-patlonal 
classification  standards  after  October  1984. 


METHOD 

Scores  of  the  H08C  sample  on  tbe 

yielded  two  seta  of  „p„l,tlon  and  (2)  standard  «:ores 

::?«rnoertr tbrifso^tutb  Pop.Utlon.  Mean  anbteat  standard  acorea  were 
computed  for  these  two  sets  of  scores. 

The  subtest  s^tndar?tcore^tuS’'were*  converted  to  Army 

aptitude  area  composites,  inese  srauaai-u 
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Uorld  War  II  and  NORC  norms.  Mean  composite 
standard  scores  the^rld  for  Arithmetic  Reasoning 

.core.  (PC),  «.a 

‘th^AlQI  Sx!‘*Se!r^QT  both 

norms* 

arores  were  computed  and  examined  for  each  of  six 
The  composite  scores  we  P  black  males,  (3)  Hispanic 

sex/ethniclty  subgroups:  (1)  iod  (6)  Hispanic  females. 

male.,  (*)  white  ““I'* •  ^''“„r«c«patloaal  claaslflo.tlon 

Finally,  the  scores  at  each  of  the  current  occp  compare  the 

cutpoints  for  the  ten  aptitu  ®  .  Che  two  different  norm  groups, 

diffeteeces  i^or«  asaJ^S  h^  the  U  not.a  differed 

To  the  extent  that  the  ability  of'the  historical  norm  group, 

',re«rons1i“a5^u.Tirg%hrc?rrent  enllatment  and  occupational 
classification  standards  would  arise. 


RESULTS  AND  DISCUSSION 

ttlTl  ^'^rlnformatl^'S  ^eporteS  “^apMcaUrat^Flgure  1. 

T^hr^earNORciassIgned  verbal,  math  and^peeded  ^  we-^lo^wer 

than  the  World  War  O^rationf  (N0)  8core  of  50  using  the  new 

examinee  receiving  a  Numerica  p  World  War  II  subtest  norms. 

TeLrinel  ««ivtrg  I  sO^n  Mathematics  Knowledge  (MK)  u«i-8  the  new 
norr^uW  Sue  recflued  a  52  using  the  historical  converalons. 

The  converse  is  true  the  ^echnical  subtest s,^Auto-Shop^^^ 

ri^strscr.;  r  aX  H  z-Ji 

rcreSrcosnSt"  -leTMrr  • 

reSs:J'"g;:priSiranrg£r  sr 

Clerical  (CL),  General  Technical  ‘  3  score  at  the  51st 

tradltlonal^orm  scores.  ^^3  ^„ld  u  at  the  52nd  ^tcentlle  on 

SrSil  'oSs  ForSl  remainder  of  the  composites,  the 
Lposite  score  exceeded  the  anSSy'stlSard  Score 

XrZ  pTetdlStriaTyNsrS  rstandard  score  points  on  Mechanical 
Maintenance  (MM) • 

Table  5  presents  the  mean  ^"“‘thrMoSc!  3„d 

sex/race  suhgtoupa.  /’'*  “  acotes.  In  all  cases  the  historical 

World  War  Il-assigned  aptitude  area  scores 


A 


subgroup  patterns. 

T..le  *  contains  “'„?"ale“nr“  t^aSuL^r 

to  be  implemented  to  maintain  abili  y  currently  in  place. 

?:tirrrtoras  «okc  .ptuna. cuto»  scotas 

rounded  to  five-point  score  intervals. 
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Table  1 


MEAN  SD6TEST  STANDAED  SCORES 
NORC  and  WWII  tbrms 


Subtest 

NORC 

WWII 

GS 

49.90 

49.63 

AR 

50.01 

50.33 

WK 

49.98 

50,81 

PC 

50.04 

51.47 

NO 

50.04 

51.14 

CS 

49.98 

52.89 

AS 

49.97 

46.26 

MK 

49.96 

51.84 

HC 

49.91 

47.55 

El 

49.95 

47.98 

7 


Table  2 

MEAN  APTITUDE  AEEA  COMPOSITE  SCOPES 
KORC  AND  WII  NORMS 


AA 

NORC 

WWII 

SUBTESTS 

CO 

99.99 

96.4 

AR,  AS,  MC,  CS 

FA 

99.99 

99.58 

AR,  MK,  MC,  CS 

EL 

100.0 

98.07 

AR,  K,  MK,  GS 

OF 

100.0 

95.53 

NO,  VE,  MC,  ^ 

SC 

100.04 

98.37 

NO,  CS,  VE,  ^ 

MM 

100.02 

93.77 

NO,  El,  MC,  ^ 

GM 

100.0 

95.81 

MK,  K,  GS,  ^ 

CL 

100.0 

101.23 

NO,  CS,  VE 

ST 

100.0 

98.39 

VE,  MK,  GS 

GT 

100.0 

100.19 

VE,  AR 

AFQT 

50.65 

52.13 

VE,  AR,  NO 
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Table  3 

MEAN  APTITUDE  AREA  DIFFERENCES 
BY  RACE/SEX  SUBGROUPS 


WHITE 

MALES 

BLACK 

MALES 

CO 

3.8* 

3.57 

FA 

.48 

1.55 

EL 

1.71 

3.34 

OF 

4.23 

3.73 

SC 

1.57 

1.04 

MM 

6.13 

6.04 

GM 

4.12 

4.82 

CL 

-1.09 

-.51 

ST 

1.53 

2.43 

GT 

-.58 

• 

CO 

AFQT 

-1.28 

-1.86 

HISPANIC 

MALES 

WHITE 

FEMALES 

3.64 

3.43 

1.06 

-.10 

2.95 

1.43 

4.14 

5.1 

1.43 

1.95 

6.18 

6.52 

4.51 

3.97 

-.50 

-1.8 

2.14 

1.35 

.56 

-.23 

-1.79 

-1.49 

BLACK 

FEMALES 

HISPANIC 

FEMALES 

3.27 

3.2 

1.07 

.85 

3.58 

3.43 

3.52 

3.67 

1.47 

1.41 

5.65 

5.67 

4.86 

4.74 

-.31 

-.64 

2.26 

2.12 

.72 

.63 

-1.86 

-1.86 

*NORC  -  WWII 
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WWII  Cut 


NORC 


Table  4 

Equivalence  of  NORC  and  WWII  Aptitude  Area  Score  Cutoffs 


Off 

80 

85 

90 

95 

100 

105 

110 

115 

120 

AA 

CO 

84 

88 

93 

98 

103 

110 

114 

119 

123 

FA 

81 

85 

89 

94 

99 

104 

109 

115 

121 

EL 

83 

87 

90 

96 

100 

105 

110 

116 

122 

OF 

85 

90 

95 

101 

107 

111 

115 

119 

123 

SC 

85 

90 

94 

98 

103 

108 

112 

115 

120 

MM 

87 

92 

96 

102 

108 

113 

117 

121 

125 

GM 

84 

89 

94 

99 

104 

110 

114 

118 

124 

CL 

83 

88 

91 

96 

101 

104 

108 

112 

116 

ST 

82 

87 

91 

96 

101 

107 

111 

116 

122 

GT 

81 

86 

90 

96 

101 

105 

110 

115 

120 

CUTOFF 
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WWII  Cut 


NORC 


Table  5 

Equivalence  of  Rounded  NORC  and  WWII  Aptitude  Area  Score  Cutoffs 


Off 

80 

85 

90 

95 

100 

105 

110 

115 

120 

AA 

CO 

85 

90 

95 

100 

105 

110 

115 

120 

125 

FA 

80 

85 

90 

95 

100 

105 

110 

115 

120 

EL 

85 

85 

90 

95 

100 

105 

110 

115 

120 

OF 

85 

90 

95 

100 

105 

110 

115 

120 

125 

SC 

85 

90 

95 

100 

105 

110 

110 

115 

120 

MM 

85 

90 

95 

100 

110 

115 

115 

120 

125 

6M 

85 

90 

95 

100 

105 

110 

115 

120 

125 

CL 

85 

90 

90 

95 

100 

105 

110 

110 

115 

ST 

80 

85 

90 

95 

100 

105 

110 

115 

120 

GT 

80 

85 

90 

95 

100 

105 

110 

115 

120 

CUTOFF 


Figure  3 
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MEMORANDUM 

for: 


ADCSPER 


DCSPER 


J 

DAPE-MPA-CS 

DIRECTORATE 

ACTION  OFFICER 

Mr.  Ruberton 

TELEPHONE  NO. 

50836/30  April  1984 

subject:  Status  Report  on  the  Implementation  of  the  1980  Youth  Pop'ulation  Norms  for 
the  Armed  Services  Vocational  Aptitude  Battery 

n  INFORMATION  '  _ □  APPROVAL  _ □.  SIGNATURE  □  INITIALS 

suspense:  sgs  osa  -•  »osd 

COMMENT  1 

FACTS : 

a.  Since  194A  the  reference  population  for  all  military  selection  and 
classification  instruments  has  been  defined  as  all  males  serving  in  the 
military  during  World  War  II  (WWII). 

b.  In  February  1980  Congress  questioned  the  appropriateness  of  interpreting 
the  ability  of  today's  youth  in  terms  of  the  norms  established  on  the  World 
War  II  group.  Documentation  appears  as  Tab  C. 

c.  In  July  1980  OSD  informed  Congress  that  it  was  participating  in  the 
Profile  of  American  Youth  Study.  The  OSD  research  was  designed  to  measure 
the  vocational  abilities  of  a  nationally  representative  sample  of  18-23 
years  olds  and  to  develop  a  new  reference  population  for  the  ASVAB.  See 
'documentation  at  Tabs  C  and  D. 

d.  In  October  1982  OSD  decided  to  adopt  the  1980  Youth  Population  as  the 
new  reference  population  for  ASVAB  beginning  1  October  1984. 

FINDINGS: 

a.  By  OSD  definition,  AFQT  Categories  I-IIIA  should  represent  the  top  50% 
of  the  military-eligible  population. 

b.  Fifty-three  percent  of  the  1980  Youth  Population  are  classified  in  AFQT 
Categories  I-IIIA  when  the  World  War  II  norms  arc  applied  (raw  score  of  76 
or  above).  See  Tab  A  and  Army  Research  Institute  Working  Paper  at  Tab  B. 

c.  The  AFQT  raw  score  which  divides  the  1980  Youth  Population  into  the 
upper  and  lower  50  percent  is  78. 
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DIRECTOR 
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M\C  ACTION :  □  STAMP 


a  DISPATCH 


OPCSPER  FORM  295 
26  reBRXJXftY  1969 


INTRAOFFICE  CONTROL 


PREVIOUS  EDITION  OF  THIS  FORM  IS  OBSOLETE. 


DISCUSSION: 


a.  In  order  to  achieve  an  AFQT  percentile  score  of  50  using  the  VWII  norms 
an  examinee  is  required  to  obtain  a  raw  score  of  76.  Implementing  the  OSD 
decision  to  use  the  1980  Youth  reference  population  means  that  an  examinee 
will  have  to  have  a  raw  score  of  78  in  order  to  achieve  the  same  AFQT 
percentile  score  of  50.  Similar  adjustments  have  been  made  at  each  of  the 
.AFQT  Category  boundaries’* 

b.  The  transition  from  the  WWIl  to  the  1980  reference  popu la t ion  al s o 
affects  the  ’Army  Aptitude  Area  composite  scores.  In  order  to  maintain 
current  aptitude  levels,  DCSPER  has  redefined  minimum  aptitude  area 
composite  requirements  for  some  MOS .  These  tables  are  at  Tabs  F  and  G,. 

c.  In  addition,  the  transition  will  affect  examinees  tested  with  ASVAB  5, 

8,  9,  and  10  who  wish  to  enlist  or  reenlist  on  or  after  1  October  1984. 

IMPACT:  _  •  .  . 

a.  Fewer  Army  applicants  will  be  classified  in  AFQT  cat egor ies ‘ 1- II lA  when 
the  1980  Youth  Population  norms  are  implemented.  For  example,  48. IZ  of 
FY83  Army  applicants  were  classified  in  AFQT  Categories  I-lIIA  using  the 
World  War  II  norms  while  43.4%  would  have  been  in  these  categories  using 
the  1980  Youth  Population  norms. 

b.  The  aptitude  area  standards  will  have  to  be  adjusted  in  Army  MOS’  with 
CO,  OF,  GM,  or  MM  aptitude  area  prerequisites.  These  tables  are  at  Tab  G. 

c.  Policies  for  all  the  Services  pertaining  to  use  of  ASVAB  5,  '8;  9,  and  10 
scores  after  1  October  1984  are  at  Tab  H. 

RECOMMENDATION:  Recommend  the  above  changes  in  the  Army  Aptitude  Area 

standards  be  approved.  These  changes  are  detailed  at  Tab  G. 
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ACTION  OFFICER 


TELEPHONE  NO.  50836/30  April  1984 


f  I  INFORMATION 


SUSPENSE : 


I  I  APPROVAL  □  SIGNATURE  □  INITIALS 


tOSD 


COMMENT  1 


FACTS: 

a.  New  forms  of  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  will 
be  implemented  throug.hout  MEPCOM  on  1  October  1984  .  The  new  forms  are 
ASVAB  11/12/13/14. 

b.  Two  Army  composites  are  recommended  for  revision  on  1  October  1984. 

These  are  the  Clerical  (CL)  and  Survei llance /Communicat ions  (SC) 
compos i t es  . 

c*  The  recommended  composites  are; 

CL=VE+AR+MK 

SC=VE+AR+AS+MC 

These  changes  are  supported  by  extensive  research  conducted  for  the  DCSPER 
by  the  Army  Research  Institute.  This  research  indicates  that  the 
recommended  composites  will  yield  better  prediction  of  performance  than 
the  current  composites.  Additional  information  about  the  research 
appears  at  Tab  E* 

RECOMMENDATION:  Recommend  approval  of  the  revised  CL  and  SC  composites 

described  above* 


1 


COORDINATION 

PERI-RS 


M\c  action; 


APPROVALS 


DIVISION  CHIEF 

DIRECTOR 

EXECUTIVE 


INITIAL  DATE 


\  I  STAMP 


□  DISPATCH 


ACTION  taken: 

ADCSPER  DCSP 

CONCUR  I— - 

APPROVED  _ _ _ 

NOTED  _ _ _ 

SEE  ME _ 

BRIEF  _ _ 


QOCSPER  FORM  295 
I»e9 


PREVIOUS  EDITION  OF  THIS  FORM  IS  OBSOLETE. 


INTRAOFFICE  CONTROL 


Distribution  of  AFQT 
Raw  Scores 


CHANGING  THE  ASVAB  REFERENCE  POPULATION 


•  Traditional  definition  of  AFQT 
CAT  I-IIIA^  AT  OR  above  50th 
Percentile  (raw  score  76) 


•  Of  the  1980  Reference  Population, 
53%  SCORE  AT  OR  ABOVE  AFQT  RAW 
SCORE  76 


•  Of  the  1980  Reference  Population, 
50%  SCORE  AT  or  above  AFQT  RAW 
SCORE  78 


•  Of  the  1980  Reference  Population, 
3%  SCORE  BETWEEN  AFQT  RAW  SCORES 

76  AND  78 


•  Of  the  FY83  Army  Applicants,  A. 7% 

SCORE  BETWEEN  AFQT  RAW  SCORES  76 
•AND  78 

•  A, 7%  WERE  AFQT  CAT-IIIA;  Would  be 
AFQT  CAT  1 1  IB  . 
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THE  1980  YOUTH  POPULATION  NORMS:  ENLISTMENT 
AND  OCCUPATIONAL  CLASSIFICATION  STANDARDS  IN  THE  ARMY 


Karen  J.  Mitchell 
Lawrence  M.  Hanser 


INTRODUCTION 

Since  1944  the  cognitive  ability  of  military  applicants  has  been 
referenced  statistically  to  the  population  of  adult  males  tested  during 
World  War  II.  The  Department  of  Defense  and  Congress  have  recently 
questioned  the  appropriateness  of  using  the  norms  established  on  the  World 
yar  II  reference  population  as  a  metric  for  interpreting  the  ability 
of  today's  applicants.  In  1979  a  decision  was  made  to  examine  the 
distribution  of  cognitive  ability  of  today's  18-23  year  olds.  An 
understanding  of  the  distribution  of  cognitive  ability  of  the  current 
military-eligible  population  would  allow  policy  makers  to  establish 
entrance  standards  and  accession  quotas  that  are  more  compatible  with 
present  manpower  resources. 

In  1980  the  Department  of  Defense,  in  cooperation  with  the  Department 
of  Labor,  sponsored  the  Profile  of  American  Youth  Study.  This  study  was 
designed  to  assess  the  vocational  aptitudes  of  a  nationally  representative 
sample  of  18-23  year  olds  and  to  develop  new  national  norms  for  the 
Department  of  Defense  enlistment  test,  the  Armed  Services  Vocational 
Aptitude  Battery  (ASVAB).  The  ASVAB  is  composed  of  eight  power  and  two  • 
speed  tests.  Four  ASVAB  subtests  are  combined  to  form  the  Armed  Forces  •. 
Qualification  Test  (AFQT),  a  general  measure  of  trainability  and  the. 
primary  criterion  for  enlistment.  Other  subtest _ combinations  are  used  in 
occupational  classification.  A  complete  discussion  of  the  content  and 
structure  of  ASVAB  may  be  found  in  Kass,  Mitchell,  Grafton  &  Wing  (1982). 

The  National  Opinion  Research  Center  (NORC)  of  tte  University  of 
Chicago  contracted  with  the  Department  of  Defense  to  administer  ASVAB  from 
July  through  October  1980  to  a  nationally  representative  sample  of  •  • 

approximately  12,000  men  and  women.  The  sample  was  constructed  to  •  ^ 

represent  age/sex/ethnicity  variables  and  major  census  regions.  The  data 
were  used  to  compare  the  1980  youth  population  to  the  World  War  II 
reference  population  and  to  FY8r  military  accessions'.  Subgroup '  comparisons 
were  conducted  on  the  basis  of  age,  sex,  race/ethnicity,  level  o*  ^  ^  • 

education,  socioeconomic  status,  and  geographic  region.  This  work  is ^ 
described  in  "Profile  of  American  Youth:  1980  .Nationwide  Administration  of 
the  Armed  Services  Vocational  Aptitude  Battery,"  March  1982.  The  data  were 
also  used  to  establish  and  verify  new  score  scales  for. the  ASV'AB  subtests 
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and  aptitude  area  composites  (Kaier  &  Sims,  1982;  Ree,  V’alentine  &  Earles, 
1983). 

Early  in  1983  Sims  and  Maier  reported  apparent  score  pattern 
discrepancies  on  the  speeded  subtests  between  the  male  NORC  -youth  sample 
and  a  series  of  male  military  applicants.  The  discrepancy  resulted'.in 
lower  scores  for  the  1980  sample  -males  on  the  two  speeded  subtests  when 
compared  to  a  sample  of  male  military  examinees .  An  analysis  of  pqssible 
sources  of  the  discrepancy  revealed  that  there  were  format  differences 
between  answer  sheets  used  in  the  NORC  field  test  and  operational  answer 
sheets  used  for  military  entrance  testing. 

In  August  of  1983,  Earles,  Guiliano,  Ree  &  Valentine  outlined  a  plan 
for  estimating  the  differences  in  performance  on  the  speeded  subtests 
attributable  to  the  non-standard  NORC  answer  sheet .  A  study  was  conducted 
in  the  fall  of  1983  at  the  Military  Enlistment  Processing  Stations  to 
develop  corrections  for  the  NORC  speeded  subtests  and  to  adjust  the 
percentile  conversion  tables.  The  corrected  subtest  and  composite  norms 
became  available  to  the  Service  Labs  in  January  of  1984  .  They  will  be 
implemented  in  October  1984  with  the  introduction  of  ASVAB  forms  11/12/13. 


PROBLEM 

Since  1944  various  forms  of  AFQT  and  ASVAB  have  been  calibrated  to  the 
World  War  II  mobilization  population.  The  use  of  a  constant  reference 
pppulation  allows  the  Services  to  (1)  track  the  quality  of  applicants  and 
recruits  over  time,  and  (2)  maintain  enlistment  and  occupational 
classification  standards  across  test  forms  and  over  time.  With  the 
introduction  of  a  new  normative  base  for  ASVAB,  research  is  needed  to 
document  the  comparability  of  applicant  and  accession  quality  based  on  the 
World  War  II  and  NORC  reference  groups.  Data  are  also  needed  on  the 
appropriateness  of  maintaining  current  enlistment  and  occupational 
classification  standards  after  October  1984.  .  — 


METHOD 

Scores  of  the  NORC  sample  on  the  power  subtests  were-  weighted  and 
converted  to  subtest  standard  scores  using  both  the  World  War  II  and  NORC 
norms.  The  NORC  sample  raw  scores  on  the  speeded  subtests  were  converted  , 
to  equated  raw  scores  based  on  the  January  1984  tables  and  then  converted 
to  subtest  standard  scores  using  both  sets  of  norms.  This  procedure 
yielded  two  sets  of  subtest  standard  scores  for  the  NORC  sample:  (1) 
standard  scores  referenced  to  the  Wll  population  and  (2)  standard  scores 
referenced  to  the  1980  Youth  Population.  Mean  subtest  standard  scores  were 
computed  for  these  two  sets  of  scores. 

The  subtest  standard  spores  were  then  combined  to  form  the  ten  Army 
aptitude  area  composites.  These  standard  score  sums  were  converted  to  Army 
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standard  scores  using  the  World  War  II  and  NORC  norms.  Kean  composite 
scores  were  computed  for  each.  Subtest  raw  scores  for  Arithmetic  Reasoning 
(AR),  Word  Knowledge  (\;K) ,  Paragraph  Comprehension  (PC),  and  Numerical 
Operations  (NO)  subtests  were  also  summed  and  the  sum  converted  to 
percentiles  for  the  AFQT  index.  Mean  AFQT  scores  were  computed  using  both 

norms . 

The  composite  scores  were  computed  and  examined  for  each  of  six 
sex/ethnicity  subgroups:  (1)  white  males,  (2)  black  males,  (3)  Hispanic 
males,  (4)  white  females,  (5)  black  females,  and  (6)  Hispanic  females. 
Finally,  the  scores  at  each  of  the  current  occupational  classification 
outpoints ‘for  the  ten  aptitude  areas  were  examined  in  order  to  compare  the 
differences  between  scoring  tables  based  on  the  two  different  norm  groups. 
To  the  extent  that  the  ability  scores  assigned  by  the  new  norms  differed 
from  the  ability  scores  assigned  on  the  basis  of  the  historical  norm  group, 
questions  of  adjusting  the  current  enlistment  and  occupational 
classification  standards  would  arise. 


RESULTS  AND  DISCUSSION 

Mean  subtest  standard  scores  using  the  NORC  and  World  War  II  norms 
appear  at  Table  1.  This  information  is  reported  graphically  at  Figure  1 . 
The  mean  NORC -assigned  verbal,  math,  and  speeded  subtest  scores  were  lower 
than  the  World  War  Il-converted  means.  This  means,  for  example,  that  an 
examinee  receiving  a  Numerical  Operations  (NO)  score  of  50  using  the  new 
norms  would  have  received  a  51  in  terms  of  the  World  War  II  subtest  norms. 
An  examinee  receiving  a  50  on  Mathematics  Knowledge  (MK)  using  the  new 
norms  would  have  received  a  52  using  the  historical  conversions.  , 


The  converse  is  true  for  the  technical  subtests,  Auto-Shop  Information 
(AS),  Mechanical  Comprehension  (MC),  and  Electronics  Information  (El).  The 
assigned  NORC  score  was  from  two  to  four  points  higher  than  the  World  War 
ll-assigned  score.  An  applicant  receiving  a  score  of  50  on  AS  would  have 
received  a  score  of  A6  using  the  World  War  II  norms.  The  mean  General 
Science  (GS)  scores  using  the  two  conversions  were  roughly  equivalent. 

Mean  aptitude  area  composite  scores  appear  at  Table  2.  The  data  are 
presented  graphically  at  Figure  2.  The  mean  NORC-as signed  composites  for^ 
Clerical  (CL),  General  Technical  (GT),  and  AFQT  were  lower  than  the 
traditional  norm  scores.  Thus,  an  applicant  receiving'  a  score  at  the  51st 
percentile  on  AFQT  using  the  1980  norms  would  be  at  the  52nd  percentile  -on 
the  WWII  norms.  For  the  remainder  of  the  composites,  the  NORC-assigned 
composite  score  exceeded  the  World  War  Il-converted  score.  The  mean 
aptitude  area  differences  ranged  from  one  half  of  an  Army  Standard  Score 
point  on  Field  Artillary  (FA)  to  six  standard  score  'points  on  Mechanical 
Maintenance  (MM)  .  _ 

Table  3  presents  the  mean  aptitude  area  score  differences  for  the 
sex/race  subgroups.  The  entries  are  the  differences  between  the  NORC-  and 
World  War  Il-assigned  aptitude  area  scores.  In  all  cases  the • histories  1 
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”  «  contains  the  »f '^ulvlletr”  tr.^iet— " 
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Table  1 


KEAN  SUBTEST  STANDARD  SCORES 


NORC  and 

WII  Norms 

X 

Subtest 

NORC 

WII 

- 

X 

SD 

X 

SD 

GS 

49.90 

10.01 

49.63 

9.70 

AR 

49.96 

10.01 

50. -33 

10.25 

WK 

49.98 

9.96 

50.81 

10 .05 

PC 

50.04 

10.04 

.51.47 

9.66 

NO 

50.04 

9.98 

51.14 

10.33 

CS 

49.98 

10.01 

52.77 

10.39 

AS 

49.97 

10,00 

46.26 

99.23 

MK 

50.01 

9.99 

51.84 

10.77 

MC 

50.00 

10.04 

47.55 

9.55 

El 

49.97 

10.00 

47.98 

.  9.86 
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Table  2 

MEAN  APTITUDE  AREA  COMPOSITE  SCORES 
NORC  AND  WWII  NORM? 


NORC 

WWII 

SUFBTESTS 

X 

SD 

X 

SD 

CO 

100.00 

20.01 

96.4 

19.94 

AR,  2^,  CS 

FA 

100.01 

20.00 

99.58 

20.53 

AR,  MK,  CS 

el 

99.99 

20.00 

98.08 

21.00 

AR,  E^,  MK,  GS 

OF 

100.0 

20.00 

95.55 

19.89 

NO,  VE,  MC,  ^ 

SC 

99.99 

20.00 

98.37 

20.16 

NO,  CS,  V-E,  AS 

MM 

100.00 

19.99 

93.78 

20.06 

NO,  El,  MC,  AS 

GM 

100.0 

20.00 

95.81 

20.49 

MK,  El,  GS,  AS 

CL 

100.0 

20.01 

101.24 

21.39 

NO,  CS,  VE 

ST 

99.99 

20.00 

-  98.39 

21.71 

VE,  MK,  GS 

GT 

100.0 

20.01 

100.19 

■  21.41 

VE,  AR  ■  •  • 

AFQT 

50.65 

29.14 

52.13 

28.29 

VE,  AR,  ND 

NOTE:  The  CL  and  SC  aptitude  area  composites  will  undergo  revision  when  ASVAB 
11/12/13  is  implemented.  Results  reported  here  pertain  to  the  8/9/10^ 
CL  and  SC  composites 
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Table  3 

MEAN  APTITUDE  AREA  DIFFERENCES 
BY  RACE/ SEX  SUBGROUPS 


M 

WHITE, 

MALES 

BLACK 

MALES 

HISPANIC 

MALES 

WHITE 

FEMALES 

BLACK 

FEMALES 

HISPANIC 

FEMALES 

CO 

3.82^ 

•  3.61 

3.67 

3.45 

3.29 

3.25 

FA 

.53 

1.59 

1.12 

-.08 

1.11 

.91 

EL 

1.69 

3.34 

2.95 

1.42' 

3.56 

3.41 

OF 

4.19 

3.71 

4.15 

5.08 

3.52 

•  3.65 

SC  ^ 

1.53 

1.03 

1.38 

1.88 

1.45 

1.37 

MM 

6.11 

6.04 

6.16 

6.51 

5.65 

5.70 

GM 

4.12 

4.84 

4.49 

3.96  • 

4.87 

4.77 

CL^ 

-1 .07 

-.48 

-.48 

GO 

r— 1 

1 

-.28  . 

-.65 

ST 

1.53 

2.42 

2.18 

1.34 

2:24'' 

2.13 

GT 

-.57 

.80 

.58 

-.23 

.68 

.59 

AFQT 

-1.28 

-1 .86 

1 

VO 

-1.49 

-1.86 

-1.86 

®  NORC  -  WIl' 


NOTE:  The  CL  and  SC  aptitude  area  composites  will  undergo  revision 
when  ASVAB  11/12/13  is  implemented.  Results  reported  here 
pertain  to  the  8/9/10  CL  and  SC  composites: 
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Table  4 

Equivalence  of  NORC  and  Aptitude  Area  Score  Cutoffs 


WII  Cut 

Off 

80 

85 

90 

95 

100 

ID  5 

110 

115 

120 

AA 

■ . 

W 

84 

88 

93 

98 

103 

110 

llii 

119 

123 

FA 

81 

85 

89 

94 

99 

104 

109 

115  • 

■  121 

EL 

83 

87 

90 

96 

100 

105 

no 

116 

.  122 

OF 

85 

90 

95 

101 

106 

111 

115 

119 

122 

NORC 

SC^ 

81 

85 

89 

94 

99 

103 

108 

114 

119 

CUTOFF 

K>'. 

87 

92 

96 

•102 

108 

113 

116 

■  121 

125 

GH  - 

84 

89 

94 

99 

104 

110 

114 

ns 

124 

a 

CL 

81 

84 

88 

92 

99 

103 

108 

114  • 

120 

ST  ■ 

82 

87 

90 

96. 

101- 

107 

no 

116. 

122 

GT 

81 

86- 

90 

96 

101 

105 

no 

115 

120 

^NOTE:  The  CL  and  .SC  aptitude  area  composites  will  undergo  revision  when  ASVAE  ll/U/13 

is  implemented.  Results  reported  here  pertain  to  the^  11/12/13  CL  and  SC 
composites.  The  new  composites  are:  CL=VE+AR+MK,  and  SC=Vfl+AR4*AS:fMC 
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Table  5 


( 

Ecui valence  of 

Rounded 

KORC  and 

II 

Apt itu  de 

Area  Sco 

r  e  Cut  c I 

fs 

WII  Cut 

Off 

80 

.  85 

90  • 

95 

100 

105 

110 

115 

120 

kA 

CO 

85 

90 

95 

100 

105 

110 

115 

%■ 

120 

• 

125 

FA 

80 

85 

90 

95 

100  ■ 

105 

no 

115 

120 

EL 

85 

85 

90 

95 

100 

105 

no 

115 

120 

OF 

85 

•  90 

95 

100 

105 

110 

.115 

120 

120 

NORC 

a 

SC 

80 

85 

90 

95  . 

100 

105 

’•  110 

115 

120 

CUTOFF 

MM  ' 

85 

90 

95 

100 

110 

115' 

115 

I’zo 

125 

GM 

'85 

90 

95 

100 

105 

110 

115 

120 

125 

a 

CL 

80 

85 

90 

.95 

100 

105 

no 

115 

120 

ST 

80 

85 

90 

95 

100 

105 

no 

115 

120 

GT 

80 

85 

90 

95 

100 

105 

no 

115 

120 

‘note:  The  CL  and  SC  aptitude  area  composites  will  undergo  r^ision  when  ASVM 

11/12/13  is  implemented.  Results  reported  here  pertain- to  the  11/12/13 
and  SC  composites.  The  new  composites  are:  CL=VE+AR4-MK , 


1/12/13  CL 
nd  SC=VE+AR+AS+MC 


Figure  1 

MEAN  SUBTEST  SCORES 
NORC  AND  WWII  NORMS 
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LEGEND:  SAMPLE  NORC  WWII 


Figure  2 

MEAN  APTITUDE  AREA  SCORES 
NORC  AND  WWII  NORMS 
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legend:  sample  norc  wwii 


Figure  3 

AVERAGE  COMPOSITE  DIFFERENCES 


EXTRACT  FROM  OASD  (M,RA&L) 
MEMORANDUM  FOR  RECORD,  DATED  15  DEC  82 
PERTAINING  TO  THE  ASVAB  STEERING  COMMITTEE 
MEETING  OF  26  OCT  82 


.  ,  of  1980  Reference  Population.  Dr.  W.  S.  Sellman  intro- 

S  fX?  Jx  Providing  historical  background.  Whan  tha  Daparcnant 

Defense  (DoD)  informed  the  Congress  in  February  1980  about  the  ASVAB 

tLr  PniJ’cr  Services  Comnittees  were  incredulous  to  learn 

In  men  test  scores  were  interpreted  against  a  reference  population  pf 

‘  ^  fficers  and  enlisted  personnel)  un'der  anas  as  of  December  31  1944- 

3uh^thncorL^of^°"^^^''  recruits  we;e  compared 

DOO^  nf  r  !  contemporary  youth  —  that  is, -.the  most  current  nationwide 
pool  of  potentially  eligible  enlistees. 

reaction,  thought  tha.t  the  delay  between  the  time  that 
A  oiscalibratipn  in  197.6  and  the  official  testimony 

►Li  DoD  kn^5  P^rt  of  a.  purposeful  deception.  They  maintained 

DoD  knew  al.  along  about  the  updated  test  results,  and  it  deliberately' 

statistical  error  in  an  effort  to  make  the  All-Volunteer  Force 
(AVF)  look  better  than  it  was  in  reality. 

Another  group  of  legislators  ~  supporters  of  the  AVF  —  thought  the  timing 

lush  iol  "ith  the  Carter  Administration^ 

push  ror  return  to  draft  registration.  The  admission  of  error  by  DoD  testing 

experts,  and  the  consequent  enlistment  of  otherwise  unqualified  recruits 

was  seen  as  an  attempt  to  buttress  the  support  for  the  registration  of  men 
ana  women* 


The  integrity  of  the  DoD  testing  program,  among  both  the  supporters  and 
detractors  of  the  AVF,  was  badly  undermined.  In  an  attempt  to  restore  s-ome 
ts  credibility  ana  to  solve  the  problem  of  obtaining  a  current  reference 
population,  DoD  administered  the  ASVAB  to  a  nationally  representative  saiole 
of  young  p^eqple  as  part  of  the  ".Profile  of  American  Youth"  study.  This,  it 
was  hoped,  would  establish  once  and  for  all  whether  the  kinds  of  people 
L  reasonably  representative  of  the  general  population, 

low-ability  individuals  so  -frequently  depicted  in  tha 
^980  and  1981  Defense  Manpower  Overview  testimony  and  in  ^ 

- -Fcport  to  the  ^use  Committee  on  Armed  Services  entitled "A-ptitude 

•'  iT^— ^ — study.. and  developing  a  new  reference  ..population  for  ini-PrprAr< 
enlistment  test  scores.  (CONTINUED  ON  NEXT  PAGET^ - - - - - - - LEil^S 


EXTRACT  FROM  OASD  (M,RA&L). 
MEMORANDUM  FOR  RECORD,  DATED  15  DETC  82 
PERTAINING  TO  THE  ASVAB  STEERING  COMMITTEE 
MEETING  OF  26  OCT  82 


^  At  its  26-28  January  1982  meeting,  the  ASVAB  Working  Group  discussed 
technical  aspects  of  using  the  "Profile”  data  to  define  a  contemporary 
reference  population  for  interpreting  ASVAB  scores.  Among  others, 
these  included  issues  such  as  the  location  of  AFQT  category  boundaries,, 
and  the  impact  of  a  new  reference  population  on  Service  qualification 
rates  for  enlistment  and* ^ob  assignment.  Obviously,  development  of  a 
new  reference  population  has  serious  implications  for  manpower  planning 
and  should  be  accomplished  only  after  careful  analysis  of  both  policy 
and  psychometric  considerations. 

In  a  27  April  1982  memorandum,  Dr.  Sicilia  asked  the  Marine  Corps  to  under¬ 
take  the  construction  of  new  norms  for  ASVAB  using  the  1980  reference 
population  data.  In  particular,  Drs.  M.  H.  Maier  and  W.  H.  Sims,  Center 
for  Naval  Analyses,  were  asked  to  develop  score  conversions  for  AFQT  and 
aptitude  composites  with  special  emphasis  on  how  a  new  reference  population 
would  affect  current  enlistment  standards  and  the  proportions  of  new 
recruits  who  would  be  placed  in  the  various  AFQT  categories. 

Following  Dr.  Sellman’s  introduction,  Dr.  M.  H.  Maier  presented  a  briefing 
on  behalf  of  the  Working  Group  which  recommended  changing  the  ASVAB 
score  scale  from  the  19A4  reference  population  to  the  1980  youth  population. 
(A  copy  of  Dr.  Maier’s  briefing  is  at  attachment  3.)'  In  particular.  Dr. 
Maier  recommended  that  the  Steering  Committee  (1)  adopt  the  1980  reference 
population,  (2)  define  the  1980  reference  population  to  consist  of  18 
to  23-year  old  males  and  females,  and  (3)  define  AFQT  category  boundaries 
to 'maintain  constant  percentile  scores  (i.e.,  the  same  boundaries  as 
currently  "'exist).  Reasons  for  recommending  the  change  included:  (1) 
the  1944  reference  population  is  almost  40  years  old  and  cannot  be  statis¬ 
tically  reproduced,  (2)  because  of  changes  in  AFQT  content,  length,  and  '  ^ 
administrative  instructions,  DoD’s  capability  to  track  current  test  scores' 
back  to  that  reference  population  is  problematic,  and‘  (3)  the  continued  / 
use  of  the  1944  reference  population  precludes  the  comparison  of  the 
aptitudes  of  new  recruits  with  those  of  the  manpower  pool  from  which 
they  are  drawn.  The.  recommendations  presented  by  Dr.  Maier  were  also 
strongly  endorsed  by  the  Defense  Advisory  Committee  on  Military  Personnel 
Testing. 


Finally,  Dr.  Maier  showed  the  scores  from  the  1980  scale  that  would  be 
required  to  maintain  the  current  enlistment  standards;  AFQT  scores 
would  require  either  no  change  or  a  one-point  change.  Aptitude  composite 
scores  for  the  Army  and  Air  Force  would  need  to  be  raised  somewhat.  ^ 


TAB  D 


EXTRACT  FROM  JULY  1980  OASD  (M,RA&L) 
REPORT  TO  THE  HOUSE  COMMITTEE' 

ON  ARMED  SERVICES 


CORRECTING  THE  CALIBRATION  PROBLEM 

The  testing  experts  employed  by  the  Department  of  Defense  and  the  out 

hired  by  the  Department  now  agree  on  the  adjustments 
^at  should  be  made  to  the  scoring  system  which  converts  raw  scores 

Department  of  Defense  began  working  on  new 
^  AoVmu  aoout  two  years  ago  as  part  of  its  coni  inuous 
o.^orL  cO  improve  the  test  and  provide  fresh  forms.  The  new  for-s 
will  be  correctly  calibrated  when  they  are  introduced.  In  ad??tion 

s-f-irr  rpdictors  o£  performance  in  miUtSry 

^  benefit,  the  availability  of  the  new  forms 
will  reduce  the  opportunity  for  test  compromis^. 

wmo^risons  capabi lity .  to' make  historic 

representative  administer  the  new  ASVAB  to  a 

-epre^.ntative  sample  of  the  nation's  young  men  and  women 

sample  «as  developed  by  the  DepartmenUof  Lab".  Thn^sis 

Research  Cente^  o^the- 

Othe  Dep«L,eJt“ub  hivf  !“  available  in  early  1981, 

f^n'^T^gfTnor■■l-■^  ...1-1.  i-L-' — ^°^...bbe_f irst  time,  the  means  to  comnare 

. ..tne^rrent  yout^pppu lotion  as  well  as  the - 

^-uYt^a  intormlTion  is  essential  foT  managing 

?  ?  tiaking-realistic  judgments  on  recruitino  results  '  ^ 

The  ?nfo^ma^?o  today's  population,  not  the  ^SArS^e. 

^  ^  needed  for  mobilization  planninq  The 

SpuI^SS^  t^  J  compare  the  characteristics  of  the  current  vouth 
Slrq^c?  manpower.  If  a  natiinaf 

be  in  a  position  ho  resumption  of  the  draft,  the  Department  should 
tJat  is  fa^r  anS  ^  cut-off  score  on  the-eStrance  test 

Dec-iqionc  i  K  manpower  needs  of  the  Military  Se-vices 

per^^tS  voLn^r'""  «ho  Lo^l^br' 

of  the  current  youth  pipuStion."  '*’=  characteristics 


TAB  E 


f 


ARI  COMMENTS  ON  EVALUATION  OF  ARMY 
AREA  COMPOSITES  •  .  - 


Tl^  lVh/vi\ 


/rKr^<v\_ 


The  oompu&jrtiun  ui  LWU  oi  the  current  AT-RHie-compoeiri^g^ 

-change  li'ith  the  1  October  1984^ii.ipl&me'nCaCian  of  ASVAB,  11/4-2 /I >r 
a/(/Ae  Army  Clerical  (CL)  and  •Surveillance/Communications  (SC)  ^  ^ 

composites^  will  be.  Tcvlaed^  the  new  &LiULLLii.‘e.o  eycr  TW'  v'g,^rr>V'KT'^v^'-'^ 
ctvv^^  ^  C  { 

CL  =  VE+AR+MK,  and 


SC  =  VE+AR+AS+MC. 

Thp  rhauge  is  supported  by  extensive  research  conducted  for 
^DCSPER^y  the  Army  Research  Institute  (ART).  Part  of  the 
research  addresses  the  predictive  validity  of  the  ASVAB  8/9/10 
Army  Aptitude  Area  Composites.  Analyses  indicate  that  the 
revised  CL  and  SC  composites  listed  above  yield -better  prediction 
of  performance  in  training  and  Army  jobs  than  do  the  8/9/10 
compositions.  While  the  research  found  no  evidence  to  support 
the  use  of  as  many  as  the  ten  composites  currently  used  by  the 
Army,  with  the  proposed  changes  in  CL  and  SC,  the  new  system  will 
perform  as  well  as  a  revised  system  with  fewer  composites. 
Further,  ARI  is  continuing  in  the  long-term  Soldier  Selection  and 
Classification  Research  (Project  A)  to  evaluate  new  psychomotor, 
perceptual,  interest,  cognitive,  and  temperament  measures  (which 
may  later  be  added  to  the  entrance  battery).  Given  .the 
difficulties  in  changing  the  current  ten-composite  system  and 
likely  substantial  improvements  from  the  ongoing  ARI  research 
project,  ARI  recommends  keeping  the  current  composite  system  with 
revisions  in  CL  and  SC  as  indicated  above.  A  report  documenting 
the  analyses  is  scheduled  for  release  by  ARI  on  15  May  1984. 


EQUIVALENCE  OF  ROUNDED  NORC  AND  WWII 

APTITUDE  AREA  SCORE  CUTOFFS  •  ,  - 

WII  CUT  OFF _ 80  85  90  95 _ 100  105  110  '  115  120 


AA 

CO 

85 

90 

95 

FA 

80 

85 

90 

EL 

85 

85 

90 

OF 

85 

90 

95 

NORC 

SC 

80 

85 

90 

CUT  OFF 

MM 

85 

90 

95 

GM 

85 

90 

95 

CL 

80 

85 

90 

ST 

80 

85 

90 

GT 

80 

85 

90 

100 

105 

110 

115 

120 

125 

95 

100 

i05  ‘ 

no 

115* 

120 

95 

100 

105 

no 

115 

120 

100 

105 

no 

115 

120 

120 

95 

100 

•  105 

no 

115 

120 

100 

110 

115 

115 

120 

125 

100 

105  . 

no 

115 

120 

125 

95 

100 

105 

no 

115 

120 

95 

100 

105 

no 

115 

120 

95 

100 

105 

no 

115 

120 

NOTE;  The  CL  and  SC  aptitude  area  composites  will  undergo  revision  when  ASVAB  11/12/13 
is  implemented.  Results  reported  here  pertain  to  the  revised  11/12/13  CL  and  SC 
composites.  The  new  composites  are:  CL=VE+AR+MK,  and  SC=VE+AR+AS+MC. 


CHANGES  IN  APTITUBE  AREA  SODRES 


CO  85  to 


COMBAT 

llBl 

llCl 

llHl 

llMl 

12B1 

12C1 

12F1 

19D1 

19E1 

19K1 

19A1 


OF  85  to 


OPR  &  FOOD 
16F1. 
16S1 
6AC1 
94B1 


90  I  90  to  951  95_to  lOOjlOO  to  105  1105  to  110  1110  to  115  1115  to  120  |  120  to  12 


90  190  to  951  95  to  100|100  to  105  1105  to  110  1110  to  115  |  115  to  120  1120  to  12 


90  to  95  I  95  to  1001 100  to  105  1  105  to  110  1  110  to  115 1  115  to  120  1  120  to  12 


GM  85  to  90 
GEN  MAINT 

41J1  41C1  41B1  OOB 

43E1  45T1  42C1  55D 

43M1  53B1  42E1 

44B1  55B1  44E1 

45B1  62G1  45D1 

51B1  68M1  45K1 

51C1  ■  45L1. 

51K1  51G1 

51M1  52C1 

51N1  52D1 

57E1  52F1 

57F1  54C1 

57H1  ■  55G1 

61F1  68J1 

62E1 
62F1  . 

.62H1 
62J1 

*SC  85  to  85  90  to  90  95  to’ 95  100  to  100  |  105  to  105  |  110  to  110|  115  to  115  |  120  to  II 


SURV/COMMO 

05B1  17C1  05C1 

31K1  •  17K1  13R1 

72E1  17.L1  17B1 

72G1  17M1 

^5  96H1  -  ,  I  " 

*No  changes  in  score  levels  between  WWII  and  1980  youth  norms. 


*Cl'  85  to  85 

90  to  90 

95  to  95 

100  to  100 

’  105  to  105 

no  -to  no 

115  to  in 

120  to  12 

CLERICAL 

|■H| 

75P1 

76V1 

7  INI 

75F1 

7  mi* 

76W1 

*■ 

73C1 

74B1 

. 

* 

75B1 

75C1 

* 

i 

i 

75D1 

[ 

■ 

75E1 

i 

76C1 

1 

76J1 

j 

76Y1 

(  - 

1 

1 

j 

1 

*EL  85  to  85 

90  to  90 

100  to  100 

!  105  to  105 

no  to  no 

115  to  115 

120  to  12 

ELECTRONICS 

25J1 

36C1 

.21G1 

23N1 

22L1 

21L1- 

26C1 

26Y1 

41G1 

36D1 

22U1 

26L1 

24E1 

24C1 

31S1 

35H1 

36E1 

23U1 

27F1 

24W1 

24G1 

36K1 

24L1 

27P1 

26D1 

24H1 

25L1 

27Q1 

31T1 

24J1 

26B1 

31VS 

32D1 

24K1 

i 

26H1 

32G1 

46N1 

•24M1 

26M1 

32H1 

24N1 

26M1 

35L1 

24P1 

26Q1  . 

35M1 

24Q1 

26R1 

35R1 

- 

24U1 

- 

26T1 

36H1 

26E1 

27B1 

36M1 

26F1 

27E1 

26K1 

27G1 

26V1 

27H1 

27N1 

27L1 

31E1 

27M1 

31J1 

31M1 

32F1  . 

31U1 

34C2 

31V1. 

34E2 

34B1 

34F2 

35E1 

34H2 

! 

35F1 

34L1 

! 

35K1 

34Y1 

41E1 

35C1 

45G1 

i  35G1 

i 

51R1 

36L1 

52G1 

93F1  j 

1 

.  7  ■' 

**  *  \ 

i 

*Ko  changes  in  score  levels  between  and  1980  youth  norms. 


*  f. 


> 


•  TAB  H 


USE  SCORES  AFTER^^l  OCT  84 


1.  After  30  September  1984,  all  applicants  in  the  DEP  will  continue  to  be  qualified 
for  enlistment  based  upon  test  scores  prior  to  1  Ocfober  1984.  Their  scores  will  be 
.converted  to  the  1980  youth  reference  population  by  MEPCOM  for  accession  reporting 
purposes, 

2.  Applicants  tested  prior  to  1  October  1984,  who  did  not  enlist  in  the  DEP  and 
desire  to  enlist  after  that  date  will  have  their  test  scores  converted  to  the  new 
1980  youth  reference  norms.  If  they  remain  qualified^,  regular  enlistment  procedures 
apply.  If  they  fail  to  qualify,  an  immediate  retest  is  authorized  which  become  the 
scores  of  record. 

3.  All  ASVAB  test  scores  for  individuals  tested  with  any  form  of  ASVAB  (5,  8,  9,  or 
10)  are  valid  for  two  years  from  original  test  date.  They  will  be  converted  to  the 
1980  norms.  Provisions  of  paragraph  2  above  will  apply  for  applicants  who  fail  to 
qualify. 

A.  In  July  1984,  a  new  form  of  ASVAB  (//lA)  will  be  introduced  solely  for  testing 
high  school  students.  This  will  replace  ASVAB  5  which  has  been  used  since  1976. 

5.  The  scores  from  ASVAB  lA  will  be  based  upon  the  1980  youth  population  norms  and 
will  be  available  for  enlistment  purposes  on  1  ‘October  1984.  They  cannot  be  used 
before  1  October  1984  because  the  new  1980  norms  for  the  reference  population  c‘annot 
be  used  before  that  time. 

6.  Students  tested  during  July  through  September  1984,  who  desire  to  enlist  prior  to 
1  October  1984,  will  have  to  be  retested  with  the  production  ASVAB  test. 

7.  Prior  service  applicants  enlisting  in  the  same  HOS  in  which  they  previously 
served  will  be  acceptable,  using  previous  scores  which  are  valid  for  5  years'. in  - 
accordance  with  AR  601-210.  Their  scores  will  be  converted  to  the  1980  iiorms  for 
reporting  purposes . 

^  .  • 

-  Those  applying  for  a  different  MOS  must  have  their  previous  test  scores 
converted  to  the  1980  reference  base,  or  retest  if  they  are  no  longer  qualified.* 

-  Those  applying  for  a  Service  other  than  the  one  in  which  they  previously 

served,  must  be  retested.  -  * 
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army  research  to  link  standards  for  enlistment  to 

ON-THE-JOB-PERFORMANCE 


ARMY  RESEARCH  OBJECTIVES 


OVERALL  ARMY  GOALS 


The  Army  Research  Institute  is  and 

large-scale,  multi-year  ^  "increase  the  overall  effec- 

classification  sys  em  *"  *  Army's  program  for  in- 

utllizetion  is  to  eosble  the  Ansj  to  i»«t  “s 

llzstion  missions  “■‘““Eh  ^°7S„sr‘'^|.!;%;/„roh  u  aimed  at 
military  occupational  /  classification  procedures 

developing  comprehensive  ‘"J  lllllilg  and  ocLpational 

to  validly  predict  performance  in  Army  training  ano 

specialties.  Specifically,  this  project  will. 


1 .  validate  ezistine  selection  measures 

and  project-developed  criteria.  The  criteria 

both  Army-wide  P®*'^®’’"'®®®®  ”®®®'^’’®gasu^«  of  MOS-specific 
oped  rating  scales  and  direct  measures  oi  nuo 

task  performance. 

2.  Develop  and  validate  new  and/or  improved  selection  and 
classification  measures. 

3  Validate  intermediate  criteria,  such  P®®^®’’"’®”®® 

training.  %^naf  "d  r.asslVnmen;  «d 

'pro“dcc?s’ion‘s'can°bc‘»ad.  throughout  an  Individual's 
tour. 

«.  Dotsrmlne  th.  relative  utility  to  the  Arm,  of  different 
performance  levels  across  MOS. 

5.  Estimate  the  relative  “^■“''■"‘'“^heir'ral 

llditranf”tiU‘’’'fo"  ■Aki'e''  operational  selection  and 
classification  decisions  a 


,  complete  description  of  the  b, 

second  year  accomplishments  have  been  P“  ^  ^  Social  Sci- 

U.S.  Arm,  Research  .l^r^Jor  ^proving  ^ 

!u::ifLVi\'n‘:ni*u\ll*lTa“tlon  o"^  Arm,  Enlisted  Personnel"  (1983). 


A  detailed  description  of  the  project  also  aPPea[s  in 

Annual  Report  to  Congress  on  Joint-Service  ^  Llnk^Stan 

dards  for  Enlistment  to  On- the- Job  Performance  (1983). 

/ 

Joint  Project  Goals 

The  Army  goals  and  the  joint  project  goals  are  the  same. 

SPECIALTIES  SELECTED  FOR  JOINT  STUDY 

The  Army's  research  focuses  on  19  MOS.  The  MOS 
to  be  representative  of  the  Army  and  include  all 
aptitude  area  composites.  Blacks,  whites, 

females  are  present  in  these  MOS  in  the  same  P^OP®^^®"®  "" 

total  accessions.  These  MOS  represent  of  annual  Army  enlist¬ 
ments. 

A  number  of  performance  measures,  includes  SS^so^ific 
training  success,  service-wide  P®’'^®/”;"®®’ ^ 
hands-on  performance,  will  be  developed  for  ^!^®®® 
sons  of  cost  efficiency,  not  all  measures  will  be  d«v«l®P®2 
the  19  MOS.  All  project  criterion  measures  will  be  developed  for 

the  following  MOS. 


1.  13B  -  Cannon  Crewman 

2.  64C  -  Motor  Transport  Operator 

3.  71L  -  Administrative  Specialist 

4.  95B  -  Military  Police 

5.  11B  -  Infantryman 

6.  19E  -  Tank  Crewman 

7.  05C  -  Radio  Teletype  Operator 

8.  63B  -  Light  Wheel  Vehicle/Power  Generation  Mechanic 

9.  91B  -  Medical  Specialist 

Measures  of  tralolne  success  end  servlce-uide  perfcrmanc 
will  be  developed  for  the  following  specialties. 

1.  76Y  -  Supply  Specialist 

2.  94B  -  Food  Service  Specialist 

3.  12B  -  Combat  Engineer 


H.  16S  -  MANPADS  Crewman 

5.  55B  -  Ammunition  Specialist 

/ 

6.  76W  -  Petroleum  Supply  Specialist 

-  -7.  54E  -  Chemical  Operations  Specialist 

8.  67N  -  Utility  Helicopter  Repairman 

9.  5IB  -  Carpentry/Masonry  Specialist 

10.  27E  -  TOW/Dragon  Repairman 


RESEARCH  DESIGN 

Types  of  Measures  to  be  Developed  and  Descriptions 

Hands-on  perfor.anc.  ...suras,  Job  knowl.dg.  tests, 

pears  in  the  FY83  Annual  Report  (1983)  and  2nd  nnu 

port  to  Congress  (1983). 

Measures  Construction 

Construction  of  the  different  measures  will  be  based  on  job 
and  task  analysis  data.  A  multi-method  job  analysis  approach  will 
employed  in  Which  requirements  are  determined  using  existing 

Army  job  lnv.ntory  procedures  and  by  ‘"'identTe'chnique 

:rjri.-rd“io"d-e.^;p';:"r?i'ri.n«  - -- 

Report  to  Congress  (1983). 

Pretesting  Strategy 

The  completed  hands-on  P*'“®%“t“pu^%‘lJe°of“^^^^^ 

lected  from  scorers  and  examinees,  me  p 
described  in  greater  detail  elsewhere. 


gampiing  Approach 


The  sampling  plan  specifies  ^ sizes*f^r^^rst- 

the  universe  of  possible  ^  j,OS  The  selection  of-  the  19 

term  enlisted  personnel  within  ”7\pfP^„ation  regarding  more 

ES"'perso...X-  (.9S3). 

Data  Collection  Procedures 

Data  collection  will  'jf aites^^lquipment ,  and  sup- 
commanders,  examination  °/  .  and  scorers,  and  a  dry  run 

f/lVstrnrP^r/ur^^^  -  s^rsSr  f^o? 

Analyses  t.o  be  Conducted 

statistical  analyses  will  focus  on  ”“«‘“.\V»io’“i«%e?l 
test  fairness  1““''  /°'' using  (D  «ct- 
forsiance  Variance  partitioning  procedures  to  estl- 

retest  procedures,  (2)  varian^^^p^^^  variables  such  as  task 

mate  generalxzability  of  interrater  agreement 

type,  scorer,  test  station,  classification  measures 

es?i;ates.  Validation  of  ®"and  confuJ^^^^  validity  of 

will  focus  on  the  job,  and  service-wide  per- 

the  battery  with  respect  !’  Js  „iii  be  used  to  examine 

formance.  Analysis-of-varian  race/ethnicity  subgroup  bias. 

C/eft°;na/y«snrTdi«u’ssed  in  full  in  the  n63  Annual  A.port 
(1983). 


CURRENT  STATUS  AND  ACCOMPLISHMENTS 


AwisA  - 

A  complete  report  of  ^rts%o*'*]ob^performance 

Army^s  re^search  tc  »“'<  "ii'^^VJ^S^’/attiJute'  (1983)  and  is 
has  been  published  by  t  y  .  , a  summary  of  accom- 

pUsimeits^Tn  "the''S?erof  job  performance  measurement  during 

CY84. 


Performance  Measure  Development 


Job  knowledge  tests,  hands  „os  13B.  64C,  71L, 
drmy-wide  rating  ”*7  describ^ing  the  rationale  for  and 
and  95B.  A  report  «»*  ^^'P^^tsit  and  selection  of  relevant  job 
procedures  followed  in  the  analysis  ana  s 


Sopu”  th“Lr,ri^.nr.el.StlPn%f  loP  t...s  for  oth.r  HOS. 

Data  Collection  -  2  I 

FLIP  T«st-  ■'•'*  t‘hr’«“FoSsCOM°lnPtal! 

13B,  61IC,  The  flalp  test  HOS  saeple  siae  by 

lations  and  two  USAREUR  Ja®*®*  J  currently  being  ana- 

site  data  is  shown  in  Table  1.  T  concurrent  validation  and 

lyzed  to  inform  decisions  about  the  FY85  concurrent  va 

criterion  development  and  field  testing  in 


Table  1 


Field  lest  sample  Size  by  Site  and  HOS  for  71L.  95B.  6«C,  and  13B 


Site  1 

71L 

95B 

64C 

13B 

Ft.  Polk 

60 

42 

Ft .  Hood 

48 

42 

Ft.  Riley 

21 

29 

Mannheim 

lOD 

150 

Herzobase 

Total 

129 

113 

155 

150 

■  nneitndlnal  Database  The  ThV‘d‘a'taba«"  s^p! 

completed  (Wise.  Wang,  and  J  predictors  and  new 

ports  the  development  and  .  describes  the  contents 

measures  of  Army  performance.  ][  ^storaae  and  access  proce- 
of  the  database,  editing  procedures,  st^e  find  ac^^ 

^p^;::dur”'D'r 

Analyses  Conducted 

Osborn  and  Hoffman  (ISB-I)  discussed  the  "l^ve  «st  'ffec- 
tl.eness  of  hands-on  and 

coverage  of  the  task  domain  is  .  .  tests  will  not  do.  Thus, 
hands-on  measures  »"lF  were 

for  many  tasks  In  the  four  HOB  comparisons, 

constructed  fFt  ^  „,.ie  evaluation  of  the 

made  for  various  types  of  ^*®^  •  surrogates  for  hands-on 

te"n!‘’ fte'%jjer“"«umlted  the“  relative  cost  of  hands-on  and 


proficiency  tests*  ^  - 

Borman  White,  and  GasV  (1984)  examined 

die?«*faSJrsr3)  interVeVwnaT/^^^^ 

Olson.  Borman,  Boberaon.  and  Bosa  (1984)  rUmara' 

s'rtisri;-  C";.^a  s 

ahipa  >>««'«”>«>'  /;/‘Tl  ,*  ldi‘r.ff.cti.aneaa. 

KSS'3.=r«n:=^^ 

cords  (reports  of  AWOL,  Articles  15.  Certificates 

tlon)  bav.  aufficlent  .arlanc.  •’’t  'H  "and  ln-a.r»i« 

rant  consideration  in  forming  cr  ..haea  indices  within  a 

of  soldier  effectiveness;  2)  to  combine  th  J  J 

model  of  soldier  effectiveness;  and,  3)  to  loentiiy 

archival  sources  it  is  roost  feasible  to  obtain  them. 

milestone  schedule 

The  criterion  instruments  for  each  HOS  are  teing 
brl.roul  VoUo..nB  tb.  n88^.U^ 

incnts  for  some  o  K^-tno  refined  and  a  decision  made 

FY84.  These  f"*J'^“"®"^L®J*is°ration^  during  the  Fy85  concurrent 
on  their  content  for  *'*“1"^®^^®:*°"  I  -11  the  MOS  are  shown  on 
validation.  These  decision  points  for  all  tne  nuo  • 

the  milestone  schedule. 


FY85  Milestone  Schedule 


FUNDING/RESOURCES 

- 

Category  FY  1985 

FY  1986 

FY  1987 

FY  1988 

6.2  l.OM 

6.3  -  5.2M 

-  6.M  78K 

;  l.OM 

JI.6M 
80K 

l.OM 

3.0M 

86K 

-  I.OM 
3.0M 
89K 
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REPLY  TO  t 

ATTENTION  OF 


PERI-RS 


1 8  APR  1983 


SUBJECT:  Study  on  Determination  of  Accession  by  Mental  Aptitude 


Assistant  Commandant 
US  Army  Infantry  School 
Ft.  Banning,  GA  31905 


1.  Last  September  you  requested  our  assistance  in  determining  whether  the 
entrance  requirements  for  MOSs  IIB,  IIC,  IIH,  and  llM  should  be  modified 
for  FY84-85.  Ve  assigned  two  scientists  from  our  Manpower  and  Personnel 
Research  Laboratory  to  assist  you,  Dr.  Larry  Hanser  and  Ms.  Darlene  Olson. 

I  understand  that  they  worked  closely  with  your  POC,  CPT  Flanagan,  to 
accomplish  this  task.  I  am  enclosing  a  copy  of  their  final  report.^  While 
1  know  that  this  information  was  presented  to  you  in  a  formal  briefing  on 
the  subject  by  CPT  Flanagan,  I  am  sure  you  will  find  it  interesting  reading. 

2.  We  are  glad  to  have  been  able  to  be  of  assistance  to  you  and  welcome  the 
opportunity  to  do  so  again.  Please  feel  free  to  call  on  us. 


1  Incl 
as 


^SIGNED 

L.  NEALE  COSBY  ' 
COL,  IN 
Commander 
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CHIEF,  S&C  TA 


PERI-RS 


■? 


SUBJECT:  Examination  of  current  Infantry  School  accessions  and 

projection  of  future  needs. 


TO:  BG  K. C.  Leuer,  Assistant  Commandant,  USAIS 

FROM:  U.S.  Array  Research  Institute  for  the  Behavioral  and  Social 

Sciences 


1.  References: 

a.  Letter,  Infantry  School,  ATSH-I-V-ED,  10  Sep  82. 

b.  Indorsement,  ARI,  PERI~RP  (10  Sep  82)  ,  subject: 

Distribution  of  Accession  by  Mental  Aptitude. 

2.  In  response  to  reference  (l.a.)  above,  the  following  report  is 
provided.  The  referenced  letter  requested  that  the  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI) 
review  the  current  mental  accession  standards  for  CMF  11,  and 
determine  whether  the  mental  ability  requirements  should  be 
changed  for  MOSs  IIB,  IIC,  IIH,  or  IIM  during  the  FY  84-85  tine 
pe  r iod . 

3.  In  order  to  clarify  the  scope  of  the  requested  Technical 
Advisory  Service  (TAS) ,  two  ARI  psycholog ists- met  with  Infantry 
School  personnel  on  November  1-2,  1982  at  Ft.  Banning.  As  a 
result  of  these  meetings  it  was  agreed  that  the  follov;ing  task 
requirements  would  be  investigated: 

a.  Examine  whether  available  data  provide  any  basis  for 
adjusting  the  current  minimum  ASVAB  Combat  Composite  Score 
(CO)  or  AFQT  Score  required  for  entry  into  an  Infantry 
military  occupational  specialty  (MOS). 

b.  Examine  whether  available  data  provide  any  basis  for 
adjusting  the  current  distribution  of  soldiers  accessed 
into  an  Infantry  MOS  across  AFQT  mental  categories. 

c.  Examine  whether,  available  data  provide  any  basis  for 
increasing  the  proportion  of  high  school  graduates  accessed 
into  Infantry  MOS. 

d.  Examine  whether  any  ability  requirements  can  be 
identified  to  aid  in  assignment  of  a  soldier  to  a 
particular  MOS  within  the  Infantry  Career  Management 
Field  (CMF).- 


e.  Examine  the  current  procedures  used  to  assign  soldiers  to 
a  particular  MOS  within  the  Infantry  CMF. 

4.  The  purpose  of  this  report  is  to  summarize  our  findings  with 
respect  to  the  above  listed  task  requirements. 

5.  ASVAB  CO  Score. 

With  respect  to  the  task  requirement  in  paragraph  3a 
above,  examination  of  the  data  indicated  that  the  Infantry 
accessed  personnel  of  comparable  ability  to  the  Army  in  general, 
and  substantially  better  in  ability  than  Infantry  personnel 
accessed  during  the  period  when  ASVAB  was  misnormed.  Evidence 
from  examining  the  relationships  between  CO  score  and  attrition 
rate  in  One-Station  Unit  Training  (OSUT)  and  performance  in  the 
field  as  reflected  by  Skill  Qualification  Test  (SQT)  scores 
indicated  that  even  a  dramatic  increase  (from  85  to  105)  in  the 
minimum  Infantry  CO  score  would  not  significantly  lower  attrition 
in  OSUT  or  substantially  increase  the  probability  of  passing  SQT. 

6.  Distribution  of  Accessions  by  Mental  Categories. 


Although  .our  database  did  not  contain  adequate 
information  to  address  all  questions  pertaining  to  the 
distribution  of  accessions  by  mental  categories  (paragraph  3b 
above)  ,  it  did  permit  us  to  consider  two  such  questions:  (1)  did 
the  distribution  of  Infantry  accessions  by  mental  category  in  FY 
1983  differ  in  any  substantial  way  from  the  distribution  of  such 
accessions  in  FY  1982,  and  (2)  how  well  are  the  Infantry 
accessions  performing  as  soldiers?  With  respect  to  the  first 
question,  we  found  improvement  in  the  distribution  of  Infantry 
accessions  into  the  higher  AFQT  categories  of  I  through  IIIA  from 
FY  1981  to  FY  1983.  In  FY81,  the  proportion  of  CMFll  accessions 
in  categories  I  through  IIIA  was  approximately  44%.  This  figure 
rose  to  approximately  52%  in  FY82,  and  as  of  the  end  of  the  first 
quarter  of  FY83  had  risen  to  approximately  60%.  With  respect  to 
the  second  question,  we  found  that  at  least  90%  of  the  Infantry 
accessions  in  the  mental  category  range  of  I— IIIA  pass  the  SQT. 
The  probability  of  passing  the  SQT  for  .category  IIIB  is  83%;  for 
category  IVA  is  77%;  and  for  category  IVB  is  72%.  Hence,  the 
Infantry  is  currently  accessing  quality  recruits  who  perform  well 
as  soldiers  after  training.  These  findings  do  notin  our 
judgment  provide  any  basis  for  adjusting  the  current  distribution 
of  Infantry  accessions- across  AFQT  mental  categories  I-IIIb,  but 
suggest  a  reduction  in  category  IV. 

7.  Education  (High  School  vs.  Non-High  School  Graduates) . 

In  response  to  the  task  requirement  in  paragraph  3c 
above,  OSUT  attrition  data  revealed  that  non-high  school 
graduates  attrited  at  over  twice  the  rate  (15.5%  vs.  6.2%)  of 


high  school  graduates.  Also,  Infantry  fir%t  terra  enlisted 
personnel  who  are  not  high  school  graduates  drop  out  of  the  Army, 
specifically  in  the  I-IIIA  and  IIIB  mental  categories,  at 
approximately  twice  the  rate  of  high  school  graduates.  While 
distributions  of  accessions  by  mental  category  showed  increased 
numbers  of  enlistees  in  Categories  I-IIIA,  many  of  these  were  not 
high  school  graduates.  Based  on  the  differentially  high  attrition 
rate  for  Infantry  accessions  v;ho  are  non-high  school  graduates  and 
the  large  number  of  them  in  CMF  11,  justification  exists  for  the 
Infantry  to  attempt  to  access  more  high  school  graduates. 

8.  Ability  Requirements  for  Infantry  MOS 

Data  addressing  the  task  requirement  in  paragraph  3d  were 
collected  from  six  company  commanders  and  3  non-commissioned  officers 
assigned  to  training  units  in  CMFll  at  Fort  Banning.  Each  of  these 
subject  matter  experts  responded  to  a  computerized  Job  Assessment 
Survey.  This  survey  is  an  experimental  instrument  developed  under 
the  direction  of  the  U.S.  Army  Research  Institute's  Systems 
Laboratory.  It  was  designed  to  aid  in  developing  a  profile  of 
abilities  required  for  successful  performance  in  a  specific  job.  For 
purposes  of  responding  to  your  request  it  was  used  as  an  aid  in 
determining  which  abilities  are  required  for  successful  performance 
in  the  various  CMFll  MOS.  It  was  also  used  to  aid  in  understanding 
how  the  ability  requirements  differ  from  one  CMFll  MOS  to  another. 

The  following  represents  a  summaryof  our  results: 

a.  Our  data  collection  found  that  in  general  the 
abilities  of  memorization  and  spatial  orientation  were 
rated  as  highly  required  for  successful  performance 

in  the  CMF  11. 

b.  In  the  IIB  MOS,  in  which  the  physical  qualificatio 
the  most  demanding  in  the  Army,  expert  judgment  (r 
corroborated  the  importance  of  such  physical  requi 
stamina;  .the. ability  to  perform  flexible,  speeded 
coordinated  muscular  actions;  and  the  ability  to  r 
quickly  to  oral  directions. 

c.  For  the  Indirect  Fire  Infantryman  (IIC  MOS)  the  profile  of 
average  requirements  showed  that  the  ability  to  use  basic 
arithmetic  operations  and  the  ability  to  make  fine 
coordinated  movements  of  the  arras,  hands,  wrists  and 
fingers  were  critical  to  successful  performance. 

d.  In  the  IIH  MOS,  highly  successful  performance  as  a  Heavy 
Anti-armor  Weapons  Crewman  was  linked  to  such 
perceptual-motor  requirements  as  the  ability  to  make 
precise  controlled  muscular  movements,  which  would 
be  necessary  to  adjust/position  equipment;  the  ability 
to  accurately  distinguish  similarities  and  differences 
when  comparing  visual  patter ns/objects;  and  the  ability 


ns  are  among 
atings) 
rements  as 
and 

espond 


/ 


to  maintain  arm-hand  steadiness. 

e.  An  examination  of  the  requirements  for  the  11^1  MOS  (Fighting 
Vehicle  Infantryman)  revealed  that  a  complex  group 
of  cognitive  and  psychomotor  abilities  were  important 
for  successful  performance.  These  requirements  included 
the  ability  to  evaluate  and- organize  information  in 
accordance  v;ith  preplanned  activities;  the  ability 
to  allocate  time  among  competing  tasks;  and  the  ability 
to  selectively  attend  to  relevant  information. 

Currently  all  CMFll  accessions  are  assigned  to  training  as 
IIX.  The  assignment  to  training  in  a  specific.  MOS  such  as  IIH  is 
made  at  the  company  level  during  BCT.  The  results  of  these 
analyses  indicate  that  different  skills  may  be  required  to 
perform  successfully  in  these  MOS.  They  also  provide  a 
preliminary  indication  of  which  skills  are  required  for  which 
MOS.  At  this  time  we  do  not  have  tests  which  adequately  measure 
all  these  abilities.  Such  tests  are  being  developed  as  part  of 
our  comprehensive  soldier  selection  and  classification  research. 

9.  Assignment  of  Soldiers  to  Infantry  MOS. 

In  response  to  the  task  requirement  in  paragraph  3e  above, 
discussions  with  Infantry  experts  (SMEs)  revealed  that  Army 
accession  quotas,  initial  company  platoon  assignments,  and 
pre-enlistment  geographic  locale  preferences  of  individual 
soldiers  determine  to  a  great  extent  assignment  to  a  specific 
Infantry  MOS.  These  initial  allocation  factors,  which  restrict 
the  available  pool  of  soldiers,  operate  independently  of  soldier 
ability  and  performance  in  the  assignment  process.  To  the  extent 
that  different  ability  profiles  are  important  in  performance  in  a 
specific  MOS,  the  present  assignment  system  will  result  in  errors 
in  assignment  and  lowered  overall  performance.  As  noted  earlier, 
assignment  of  individuals  to  specific  MOS  at  the  Infantry  School 
is  accomplished  at  the  company  level.  Under  these  circumstances 
the  commander  has  very  little  freedom  for  the  reasons  mentioned 
above.  Perhaps  more  importantly,  the  individuals  who  make  these 
assignment  decisions  have  been  provided  with  very  little  guidance 
on  how  to  make  them.  One  recommendation  for  improving  these 
conditions  would  be  to  remove  the  decision  to  the  battalion, 
brigade,  or  school  level.  This  would  provide  more- freedom  in  the 
assignment  decisions  by  enlarging  the  eligible  pool.  At  the  same 
time  arguments  could  b*e  made  to  continue  to  make  the  decisions  at 
the  company  level.  After  all,  the  company  personnel  have  the 
closest  contact  with  the  individuals  who  are  to  be  assigned,  and 
perhaps  know  them  best.  One  reasonable  answer  to  this  problem 
would  be  to  provide  broad  guidelines  to  the  company  commanders  to 
aid  them  in  assigning  individuals  to  specific  MOS.  Hence, 
although  in  terms  of  selection  the  Infantry  as  a  v;hole  is 
obtaining  quality  accessions,  policies  at  both  the  DA  and  Army 
Training  Center  level  which  impact  on  classification  or 


assignment  of  soldiers  to  specific  CMFll  MO^  should  be  examined 
in  order  to  better  optimize  the  match  between  soldiers  with 
certain  abilities  and  specific  requirements  of  an  MOS. 
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Spatial  abilities  influence  effective  adjtistment  to  the  physical 
environment  and  impact  on  how  individuals  perceive  and  interact  with  their 
environments .  Research  has  showi  that  spatial  abilities  play  crucial  roles 
in  academic  performance,  vocational  choice,  and  job  performance  (e.g., 
Ghiselli,  1966;  Fermema  &  Sherman,  1977;  McGee,  1979).  Historically, 
research  on  cognitive  abilities  has  found  that  men  tend  to  obtain  sign¬ 
ificantly  higher  mean  scores  th^  women  on  measures  of  spatial  relations, 
visualization  and  orientation  skills  (Maccoby  &  Jackson,  1974). 

During  the  last  decade  this  general  finding  of  sex-related  differences 
in  spatial  aptitude  has  been  questioned  (Fairweather,  1980;  Hyde,  1981; 

Caplan,  MacPherson,  &  Ibbin,  1985).  First,  it  has  been  argued  that  spatial 
ability  is  not  a  well-defined  construct  in  terms  of  either  subabilities  or 
psychological  process  variables  (Lohman,  1979).  Second,  findings  regarding 
vAiich  spatial  ability  factors  show  sex-related  differences,  at  which  develop¬ 
mental  ages ,  and  in  relation  to  which  spatial  tasks/  test  items  have  yet  to 
emerge  consistently.  Third,  the  advent  of  meta-analytic  techniques  (Glass, 
McGaw,  &  Smith,  1981;  Hyde,  1981)  for  examining  the  effect  size  of  mean 
differences  across  individioal  research  studies  suggests  that  the  magnitude  of 
gender  differences  for  visual- spatial  ability  is  small,  with  5%  or  less  of 
the  variance  explained  by  sex.  While  the  existence  and  importance  of  sex- 
related  differences  in  spatial  abilities  is  debated,  the  practical  signifi¬ 
cance  of  even  small  differences  between  the  sexes  in  spatial  skills  in  terms 
of  opportunities,  perceptions,  and  performance  expectations  should  not  be 
ignored. 

This  research  was  conducted  as  a  component  of  Project  A,  a  large 
selection  and  classification  research  program  initiated  by  the  Army.  The 

purpose  of  Project  A  is  to  validate  the  Armed  Services  Vocational  Aptitude 


Battery  (ASVAB) ,  the  military's  entry-level  cognitive  selection  battery,  as 
well  as  other  new  spatial,  perceptual,  psychomotor ,  and  temperament  measures 
against  job  performance  criteria.  Specifically,  the  pxjrposes  of  the  research 
reported  here  are  to:  (1)  investigate  sex-related  differences  in  mean  scores 
on  the  Project  A  paper- and- pencil  spatial  tests,  (2)  examine  correlations 
between  spatial  tests  and  ASVAB  subtests,  (3)  compare  the  factor  structures 
of  the  spatial  tests  and  ASVAB  siibtests  for  men  and  women,  and  (4)  examine 
the  correlations  between  scores  on  the  spatial  predictors  and  Project  A 
performance  constructs  for  the  sexes. 

MEIHOD 

Subj ects .  The  sample  contained  4926  (females  =  839  and  males  =  4087) 
first- term  Army  enlisted  personnel  in  9  jobs  representing  clerical/ adminis¬ 
trative,  operations,  and  skilled/ technical  military  occupational  specialties 
(M)S).  These  jobs  were  sampled  at  eleven  continental  Iftiited  States  and  four 
European  Army  installations.  Table  1  describes  the  sample  by  sex  and  Army 
job. 


Insert  Table  1  about  here 


Cognitive  Ability  Measures.  The  operational  Armed  Services  Vocational 
Aptitude  Battery  (ASVAB)  was  \ised  in  this  research  as  a  measure  of  general 
cognitive  ability.  The  ASVAB,  a  paper-and-pencil  instrunents  contains  the 
following  10  independently  timed  and  scored  subtests:  (1)  Arithmetic 
Reasoning,  (2)  Mechanical  Comprehension,  (3)  Math  Knowledge,  (4)  Numerical 
Operations,  (5)  Paragraph  Comprehension,  (6)  Auto/ Shop  Information,  (7)  Ilford 
Knowledge,  (8)  General  Science,  (9)  Electronics  Information,  and  (10)  Coding 
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speed.  The  30  item  Arithmetic  Reasoning  subtest  requires  the  ability  to 
implement  the  arithmetic  operations  of  addition,  subtraction,  multiplication, 
and  division  of  integers  and  fractions.  The  Mechanical  Comprehension 
subtest,  whieh  contains  25  items,  assesses  knowledge  of  the  properties  of 
materials  and  structures  and  how  mechanical  devices  operate.  The  Mathematics 
Knowledge  test  (25  items)  requires  the  ability  to  perform  formal  algebraic 
operations.  The  speed  with  vhich  a  respondent  can  perform  simple  arithmetic 
operations  on  primarily  single  digit  \diole  numbers  is  assessed  by  the  50  item 
Numerical  Operations  subtest.  The  Paragraph  Comprehension  subtest  presents 
respondents  with  a  brief  reading  passage  and  asks  questions  about  the  content 
of  the  passage  which  require  the  abilities  to  infer  and  generalize  from  vhat 
is  read.  The  25- item.  Auto/ Shop  Information  subtest  measures  knowledge  of 
automobile  repairs  and  facts  about  metal/ wood  shops.  The  vocabulary  of 
respondents  is  measured  with  the  35  item  Ward  Knowledge  siibtest.  The  General 
Science  subtest  measures  knowledge  of  the  physical  and  biological  sciences 
and  the  Electronics  Information  subtest  assesses  practical  knowledge  about 
electricity  and  electronics.  The  Coding  Speed  subtest,  which  draws  heavily 
on  short-term  memory  requires  respondents  to  compare  symbols  (i.e.,  a  common 
word  with  a  four-digit  number)  and  transfer  the  information  from  the  test 
booklet  to  the  answer  sheet  as  rapidly  as  possible. 

The  Project  A  spatial  battery  used  in  this  research  contains  6  timed 
paper-and  pencil  tests ,  with  a  total  of  220  itsns  and  a  time  limit  of  63 
minutes.  A  30- item  Spatial  (Figural)  Reasoning  test  was  used  to  measure 
soldiers’  ability  to  determine  the  principles  \diich  govern  the  relationships 
among  a  group  of  abstract  figures.  This  test  is  expected  to  assist  in 
prediction  of  performance  in  such  areas  as  identification  and  detection  of 
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targets,  troubleshooting  and  report  work,  and  analysis  of  intelligence  data. 
The  visualization  measures  consist  of  (1)  a  90-itQn  Object  Rotation  test, 
that  assesses  the  mental  rotation  of  two-dimensional  figures,  (2)  a  24-itan 
Maze  test,  that  measures  spatial  scanning  or  the  ability  to  identi:fy  a 
specific  pattern  or  configuration  in  a  complex  visual  field,  and  (3)  Assem¬ 
bling  Objects,  a  32- item  measvire  of  the  ability  to  visualize  assembled 
objects  from  their  component  parts.  According  to  Peterson  (1986)  these 
visualization  tests  were  expected  to  predict  success  in  map  drawing  and  tisage 
in  field  operations  related  to  mechanics  and  construction  activities.  Two 
orientation  tests  were  also  administered:  A  24-it0n  Orientation  test,  ^ich 
assesses  soldiers'  ability  to  maintain  their  perspective  or  bearing  with 
respect  to  some  object  that  has  been  rotated,  and  the  Map  test,  which 
measures  the  ability  to  orient  oneself  tasing  a  map  and  knowledge  of  compass 
directions.  The  spatial  orientation  tests  were  developed  to  predict  success 
in  activities  which  involve  maintaining  physical  position  relative  to 
envirormental  landmarks  or  location  under  frequent  directional  changes 
(Peterson,  1986).  The  spatial  tests  are  very  reliable  with  split-half 
reliabilities  corrected  for  length  using  the  Spearman-Brown  formula  ranging 
from  .87  (Reasoning)  to  .99  (Object  Rotation). 

Performance  Measures.  Criterion  development  work  was  condijcted  by  the 
Project  A  contractors  and  included  construction  of  the  following  measxires: 

(1)  Army-wide  rating  scales  relevant  for  evaluating  soldiers  in  any  first- 
tour  Army  job,  (2)  job- specific  rating  scales,  (3)  hands-on  task  proficiency 
measures,  and  (4)  job  knowledge  tests.  The  Army-wide  rating  scales  were 
developed  using  a  variant  of  the  behaviorally-anchored  rating  scale  method¬ 
ology,  and  anphasize  performance  dimensions  relevant  to  any  MDS  (e.g.. 


4 


maintaining  equipment).  The  job- specific  scales,  v^ich  were  also  7- point 
bdiavior  sunmary  scales,  focus  on  narrow  performance  areas  relevant  to  a 
designated  job  (e.g.,  transporting  personnel  for  the  motor  transport  operator 
job).  The  hands-on  tests  consisted  of  15  MOS-specific  tasks.  Hands-on 
scores  were  computed  for  each  soldier  by  averaging  the  proportions  passed 
across  the  tasks  tested.  Multiple  choice  tests  were  developed  to  assess  job 
knowledge  relevant  to  import  and  representative  tasks  in  an  MOS.  A  total  job 
knowledge  score  for  each  research  participant  was  derived  as  a  percentage  of 
the  number  of  items  answered  correctly.  Factor-analysis  of  the  performance 
ratings  resulted  in  an  interpretable  solution  with  three  factors:  (1)  Effort 
and  Leadership,  (2)  Personal  Discipline,  and  (3)  Physical  Fitness  and 
Military  Bearing  (Campbell,  Hanser,  Wise,  1986).  These  represent  the  "will 
do"  or  motivational  aspects  of  the  performance  domain.  In  addition,  a 
General  Soldiering  Proficiency  factor  and  a  MOS-specific  Core  Technical 
Proficiency  factor,  which  are  based  on  factor  analysis  of  written  job 
knowledge  and  hands-on  tasks,  emerged  as  representative  of  the  "can  do"  or 
more  job/ task  specific  components  of  performance.  Hence,  the  interactive 
research  process  pursued  in  Project  A  has  resvilted  in  a  predictor  and 
criterion  space  that  are  multidimensional. 

Specifically,  then,  the  criterion  domain  as  defined  in  Project  A 
contains  five  performance  constiructs: 

1)  The  Core  Technical  Proficiency  construct  represents  the  proficiency 
with  ^diich  the  soldier  performs  the  tasks  vJiich  are  "central"  to  his/her  MOS. 
It  refers  to  how  well  the  individual  can  execute  the  core  technical  tasks  tiie 
job  requires,  given  the  motivation  to  do  so. 

2)  The  General  Soldiering  Proficiency  construct  represents  the  degree 
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tx)  vdiich  the  soldier  can  perform  a  variety  of  Anny-wide  general  soldiering 
tasks,  incltding  use  of  basic  weapons,  NBC  procedures,  first  aid,  basic  field 
techniques,  etc. 

3)  The  Effort  and  Leadership  performance  construct  reflects  the  extent 
to  which  the  soldier  exerts  effort  over  the  full  range  of  job  tasks,  persev¬ 
eres  under  adverse  or  dangerous  conditions,  and  demonstrates  the  willingness 
to  exercise  leadership  and  support  toward  peers. 

4)  The  Personal  Discipline  Construct  reflects  the  degree  to  vhich  the 
soldier  adheres  to  Army  Regulations  and  traditions,  exercises  personal  self 
control,  demonstrates  constraint  in  daily  behavior  and  does  not  cause 
disciplinary  problems. 

5)  The  Physical  Fitness  and  Military  Bearing  performance  construct 
represents  the  degree  to  vhich  the  individual  maintains  an  appropriate 
military  appearance  and  remains  in  good  physical  condition. 

KESOITS 

Descriptive  Statistics;  Table  2  presents  the  means,  standard  devia¬ 
tions,  ^ tests  and  effect  sizes  on  the  Project  A  spatial  tests  for  total 
sample  and  by  sex  group. 


Insert  Table  2  about  here 


For  the  total  sample,  multivariate  analysis  indicated  that  there  is  a 
significant  overall  sex- related  difference  cn  the  spatial  tests  (T^  =  279.30, 
2.=  .0001).  Ihivariate  t- statistics  show  that  men  have  significantly  higher 
mean  scores  than  women  on  all  the  spatial  measxires  except  Assembling  Objects 
-1.08,  £=  .28)  and  Spatial  Reasoning  (t  =  -4.85,  2.=  .0001).  The 
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magnitude  of  the  significant  sex-related  differences  on  the  spatial  tests  is 
on  the  average,  less  than  3/4  of  a  standard  deviation.  Further,  the  magni¬ 
tude  of  the  effect  sizes  were  all  small  (based  on  Cohen's  [1977]  interpreta¬ 
tions)  ,  indicating  that  only  minimal  mean  differences  exist  between  men  and 
women  on  the  Project  A  spatial  tests. 

Correlational  Analyses.  Tables  3  and  4  present  the  intercorrelation 
matrices  for  the  Project  A  spatial  tests  and  the  Armed  Forces  Qualifications 
Test  (AFQT)  by  total  sample  and  by  sex  group.  In  the  total  sample,  AFQT  was 


Insert  Tables  3  and  4  about  here 


most  highly  correlated  with  the  Map  test  and  least  correlated  with  the  Object 
Rotation  test.  This  finding  was  also  observed  for  males  and  females  sep¬ 
arately.  Intercorrelations  among  the  spatial  tests  show  that  the  orientation 
measures  tend  to  be  more  highly  related  to  each  other  than  to  the  visualiza¬ 
tion  tests.  This  finding  lends  some  support  for  the  convergent  and  discrimi¬ 
nant  validity  of  the  separate  measures. 

Table  5  displays  the  correlations  between  the  ASVAB  subtests  and  the 
Project  A  spatial  tests  for  the  total  sample.  Generally  the  strongest 


Insert  Table  5  about  here 


relationships  to  the  spatial  measures  are  noted  between  the  ASVAB  sub  tests 
Arithmetic  Reasoning,  Mechanical  Comprehension,  and  Mathematics  Knowledge. 
This  is  particularly  true  for  the  Map  and  Spatial  Reasoning  Tests.  Further, 
in  general,  correlations  tend  to  be  higher  between  the  Map  test  and  all  ASVAB 
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subtests  except  Numerical  Operations  than  for  the  other  spatial  tests.  In 
comparison,  the  weakest  correlations  were  found  between  the  speeded  ASVAB 
subtests  (Ntmerical  Operations  and  Coding  Speed)  and  the  spatial  measxares. 

Correlations  between  the  ASVAB  subtests  and  the  Project  A  spatial  tests 
for  males  and  females  are  displayed  in  Table  6.  In  general,  slightly 


Insert  Table  6  about  here 


stronger  relationships  between  scores  on  the  ASVAB  subtests  and  the  spatial 
tests  were  observed  for  males  than  for  females,  probably  due  to  greater 
variability  among  scores  for  males.  As  previously  noted  for  the  total 
sample,  the  largest  sigiificant  correlations  for  both  sexes  were  found 
between  the  spatial  tests  (e.g. ,  Map  and  Spatial  Reasoning)  and  the  qraanti- 
tative  (e.g..  Arithmetic  Reasoning  and  Math  Kiowledge)  and  mechanical 
sijb tests  of  ASVAB. 

Factor  Analysis.  Although  overall  mean  scores  on  the  spatial  tests 
differ  by  sex,  some  commonalities  in  the  structures  were  found  for  males  and 
fanales  in  this  research.  Principal  components  factor  analyses  with  varimax 
rotation,  requesting  a  four- factor  solution,  were  run  for  the  total  sample 
and  for  each  sex  group.  These  factor  solutions  are  presented  in  Tables  7  and 
3,  For  the  total  sample.  Table  7  shows  a  strong  spatial  factor  (Factor  1), 


Insert  Tables  7  and  8  about  here 


with  all  the  Project  A  spatial  tests  loading  at  .60  or  higher.  Both  quanti¬ 
tative  and  verbal  ASVAB  subtests  load  on  Factor  2.  Factor  3  is  technical  in 
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cxmtent,  with  strong  loadings  for  Mechanical  Comprehension,  Auto/ Shop,  and 
Electronic  Information.  The  speeded  ASVAB  subtests.  Numerical  Operations  and 
Coding  speed  make  \jp  Factor  4. 

Table  8  shows  the  separate  factor  structures  by  sex  grovip.  For  females, 
Ifechanical  Compr^ension  loads  more  strongly  with  the  Project  A  Spatial 
tests.  For  males'  Mechanical  Comprehension  loads  higher  with  the  verbal  and 
technical  subtests  from  ASVAB  than  with  the  spatial  tests.  Also,  for 
fanales,  the  spatial  tests  all  load  highest  on  the  same  factor  with  Mechani¬ 
cal  Comprehension,  but  the  Map  and  Spatial  Reasoning  Tests  also  load  with  the 
quantitative  ASVAB  subtests.  In  contrast,  for  males,  the  spatial  tests  tend 
to  load  on  two  separate  factors.  Specifically,  Map,  Orientation,  and  Spatial 
Reasoning  load  with  the  quantitative  ASVAB  subtests  viiile  Object  Rotation  and 
Maze  form  their  own  factor  on  which  Mechanical  Ccxnprehension  loads  to  a 
moderate  degree.  Assemble  Objects  loads  equally  high  on  both  of  these 
factors. 

Correlations  between  Project  A  spatial  tests  and  the  five  criterion 
constructs  are  presented  for  the  total  group  in  Table  9  and  by  sex  group  in 
Table  10.  For  the  total  group,  correlations  between  the  "can  do"  constructs 
(General  Soldiering  Proficiency  and  Core  Technical  Proficiency)  are  higher 
(range  =  .18  to  .38)  than  for  the  "will  do"  constructs  (Effort  and  leader¬ 
ship,  Personal  Discipline  and  Military  Bearing  and  Physical  Fitness  (range  = 
.01  to  .12),  as  would  be  expected.  This  pattern  is  the  same  for  the  two  sex 
groups,  but  the  range  of  correlations  is  higher  for  males. 


Insert  Tables  9  and  10  about  here 
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OONCIUSION 


In  sunmary,  as  previous  research  in  spatial  abilities  has  found,  the 
findings  from  this  research  provide  support  for  significant  sex-related 
differences  an  spatial  abilities  tests  in  favor  of  males.  However,  we  also 
fomd  a  significant  spatial  difference  in  favor  of  females  on  Reasoning  and 
no  difference  on  Assemble  Objects,  a  test  of  spatial  visualization.  Although 
significant  mean  differences  do  exist,  the  magnitudes  of  the  differences  and 
their  actual  effect  sizes  are  not  even  moderate.  Further,  higher  correla¬ 
tions  among  the  Project  A  spatial  tests  themselves  and  in  relation  to  AFQT 
and  to  the  individual  ASVAB  subtests  are  found  for  males,  but  these  differ¬ 
ences  are  amall  also.  In  addition,  patterns  of  intercorrelations  for  the  two 
sex  groups  appear  more  similar  than  different.  This  is  also  true  of  the 
relationships  between  the  spatial  measures  and  the  Project  A  criterion  con¬ 
structs.  There  are  somevAiat  different  factor  structures  for  the  two  sex 
groups.  These  differences  are  not  clear-cut,  however,  given  that  some  tests 
and  subtests  have  moderate  to  high  loadings  on  more  than  one  factor. 

A  general  conclusion  from  these  findings  is  that  differences  as  a 
function  of  sex.  of  the  soldier  do  exist  with  the  Project  A  spatial  tests,  but 
that  these  differences  are  not  great.  It  remains  to  be  seen,  however, 
whether  the  differences  are  such  that  selection  based  on  the  spatial  tests 
may  be  biased  in  favor  of  one  sex  group  over  the  other.  This  important 
question  mxist  be  answered.  Analyses  to  address  this  issxie  are  presently 
underway. 
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Description  of  the  Sample 


and  Standards  for  Total  Sample,  Females,  and  Males  on  the  Project  A  Spatial  Tests 
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Armed  Forces  Qualifications  Test  (AFQT)  for  the  Total  Sample 


Qualifications  Test  (AFQT)  for  Males  and  Females 


Correlations  Between  ASVAB  Sub tests  and  Project  A  Spatial  Tests 


Note.  The  Project  A  spatial  tests  are;  AO  =  Assemble  Objects,  MP  =  Map, 

HZ~^  Haze,  OR  >  Object  Rotation,  OT  «  Orientation,  and  RS  »  Spatial  Reasoning. 
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Table  7 

Factor  Loadings  of  the  ASVAB  Subtests  and  the  Project  A  Spatial  Tests  for  the  Total  Sample 
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Table  8 

Factor  Loadings  of  the  ASVAB  Subtests  and  the  Project  A  Spatial  Tests  for  Females  and  Males 


Mote:  Principal  conponenta  factor  analysis  with  a  varlnax  rotation. 


Correlations  Between  Project  A  Spatial  Tests  and  Criterion 
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Introduction 


Beginning  in  1979  military  testing 
increasing  attention,  from  both  the  Department  of  Defense  and 
SSgSSs?  The  reasons  for  this  interest  are  varied  and  include: 


1)  The  discovery  in  1979  that  the  then  operational  selection 
test  battery  for  military  enlistment  was  misnormed. 


2)  The  switch  in  reference  groups  for  military  tests  from 
the  1944  mobilization  population  (all  men,  including 
officers,  serving  under  arms  during  that  year)  to  the  1980 
youth  population  (all  17-  to  23-year  olds  in  1980  including 
both  males  and  females) . 


3)  A  Congressional  mandate  to  link  entry  test  scores 
directly  to  on-the-job  performance  rather  than  to  tradi¬ 
tional  training  performance. 

4)  Intense  interest  in  combining  computer  technology  with 
emerging  developments  in  testing  theory. 


As  an  integrated  response  to  the  various  concerns  the  U.S. 
Army  Research  Institute  designed  Project  A,  "Improving  the 
Selection,  Classification,  and  Utilization  of  Army  Enlisted 
Personnel."  Project  A  had  many  objectives  including:  (1) 
validation  of  the  existing  enlisted  selection  and  classification 
battery,  the  Armed  Forces  Vocational  Aptitude  Battery  (ASVAB) , 

(2)  development  and  validation  of  new  cognitive  and  non-cognitive 
measures  that  add  to  the  ASVAB  in  predicting  total  soldier 
performance,  and  (3)  development  of  a  comprehensive  set  of 
performance  measures  including  paper-and-pencil  tests  of  school 
knowledge  and  job  knowledge,  hands-on  tests  of  job  performance, 
and  behaviorally-anchored  rating  scales. 


The  Project  A  research  design  consists  of  three  main  stages 
of  data  collection  and  analysis  in  an  iterative  progression  of 
development,  testing,  evaluation,  and  further  refinement  of  the 
selection/classification  instruments  (predictors)  and  measures 
of  job  performance  (criteria) .  In  the  first  stage,  individual 
level  data  from  FY  81-82  were  examined  to  explore  relationships 
between  soldiers'  ASVAB  scores  and  their  later  performance  in 
training  and  on  first-tour  Skill  Qualification  Tests  (SQT) . 


Pjclor  to  the  second  stage  of  the  data  collection,  19  Military 
Occupational  Specialties  (MOS)  were  selected  as  a  representative 
sample  of  the  Army's  250+  entry-level  jobs.  These  MOS  were 
selected  based  on  1)  a  clustering  of  MOS  based  on  similarity  of 
job  content,  2)  the  fact  that  these  MOS  are  representative  of 
Army  jobs  and  account  for  over  45%  of  the  Army's  accessions,  and 
3)  the  recommendations  of  General  Officers  representing  various 
Army  Commands  at  that  time.  Nine  of  the  19  MOS  were  then 


selected  to  have  both  paper-and-pencil  and  hands-on  tests  of  job 
performance  in  addition  to  the  school  knowledge  tests  and  ratings 
Administered  in  all  of  the  MOS.  The  nine  fully-tested  MOS  in 
Project  A  are:  Infantryman  (IIB) ,  Cannon  Crewman  (13B),  Tank 
Crewman  (19E) ,  Radio  Operator  (31C) ,  Light  Wheel  Vehi^e  Mechanic 
(63B),  Administrative  Specialist  (71L) ,  Motor  Transport  Operator 
(88M)  Medical  Care  Specialist  (91A) ,  and  Military  Police  (95B) . 


The  second  stage  of  the  Project  A  research,  the  Concurrent 
Validation  (CV) ,  was  conducted  in  FY  1985.  In  the  CV,  over  9400 
soldiers  in  the  19  MOS  were  administered  the  new  Project  A 
predictor  tests,  which  included  measures  of  spatial  abilities, 
temperament  and  vocational  interest,  as  well  as  computerized 
tests  of  perceptual  and  psychomotor  skills.  These  tests 
designed  to  expand  the  predictor  domain  in  terms  of  individual 
characteristics  and  attributes  that  might  be  important  for 
selection  in  predicting  more  aspects  of  performance.  The  new 
predictors  were  intended  to  supplement  those  cognitive  abilities 
assessed  by  ASVAB. 


Concurrently,  the  CV  soldiers,  who  had  been  in  the  Army  12 
to  27  months,  were  administered  a  comprehensive  set  of  job 
performance  measures  including  supervisory  and  peer  ratings, 
yj^itten  school  and  job  knowledge  tests,  and  MOS— specific  hands-on 
task  proficiency  measures. 

One  of  the  major  scientific  contributions  of  the  Project  A 
research  to  date  is  a  comprehensive  modeling  of  the  job  perfor¬ 
mance  domain.  The  criterion  development  efforts  in  the  project 
were  driven  by  the  idea  that  job  performance  is  multi-dimensional 
and  that  performance  is  best  measured  through  a  variety  of 
methods.  Analyses  of  the  criterion  data  collected  during  the  CV 
resulted  in  five  empirically  derived  dimensions  that  represent 
overall  Army  job  performance.  The  five  Project  A  job  performance 
dimensions  are: 


1)  MOS-Teehnlcal  Knowledge  and  Skill.  The  proficiency 
with  which  a  soldier  performs  the  technical  tasks  that  are 
critical  and  central  to  his/her  MOS. 


2)  General  Soldiering  Proficiency:  How  well  a  soldier 
executes  common  soldiering  tasks  such  as  first  aid  and  land 
navigation. 

3)  ■Rffort  apH  T.eadership;  The  degree  to  which  a  soldier 
exerts  effort  over  the  full  range  of  job  tasks,  perseveres 
under  adverse  or  dangerous  conditions,  and  demonstrates 
leadership  and  support  toward  peers. 

4)  Personal  n^scioline:  The  degree  to  which  a  soldier 
adheres  to  Army  regulations,  exhibits  self-control  and 
does  not  create  disciplinary  problems. 


5)  Physical  Fitness  and  Military  Bearing;  The  degree  to 
which  a  soldier  maintains  an  appropriate  military  appearance 
and  remains  in  good  physical  condition. 

The  first  two  performance  dimensions  involve  a  soldier's 
ability  to  perform  the  technical  requirements  of  his/her  job  and 
are  often  referred  to  as  "can  do"  factors.  The  remaining  three 
performance  dimensions  are  more  attitudinal  or  motivational  in 
nature  and  are  often  called  "will  do"  factors. 

The  third  stage  of  the  Project  A  research,  known  as  the 
Longitudinal  Validation  (LV) ,  started  in  1986  with  the  admin~ 
istration  of  the  battery  of  new  Project  A  predictor  tests  to 
approximately  55,000  new  recruits  in  21  MOS.  At  the  end  of 
his/her  Advanced  Individual  Training  (AIT)  or  One  Station  Unit 
Training  (OSUT) ,  each  soldier  was  administered  the  school 
knowledge  test  for  his/her  MOS,  and  a  set  of  rating  scales  was 
collected  from  supervisors  and  peers.  Approximately  11,000  of 
these  soldiers  were  followed  into  their  first-tour  field 
ass ignmen't  and  have  been  administered  a  set  of  MOS^specific  and 
Army— wide  job  performance  measures  during  FY  88  and  FY  89.  In 
addition,  about  1,000  of  the  soldiers  from  the  CV  sample  who 
reenlisted  have  been  assessed  on  a  battery  of  performance 
measures  appropriate  for  second-tour  including  rating  scales, 
written  job  knowledge  tests,  hands-on  measures,  and  leadership 
measures. 

Results  and  Products 

During  the  execution  of  the  research  plan.  Project  A  has  made 
many  scientific  contributions  to  the  field  of  Industrial  and 
Organizational  psychology  and  so  far  has  delivered  four  major 
products  to  the  Army. 

ASVAB.  One  product  which  resulted  from  the  Concurrent 
Validation  research  effort  involves  improvements  in  the  computa¬ 
tion  and  use  of  ASVAB  Aptitude  Area  (AA)  Composites.  These 
composites  are  used  to  determine  eligibility  for  training  in  Army 
jobs.  Beginning  with  the  operational  implementation  of  ASVAB 
11/12/13/14  in  October  1984  the  formulae  for  the  Clerical  (CL) 
and  the  Surveillance/Communications  (SC)  composites  were  changed. 
These  changes  resulted  in  both  better  accuracy  and  improved 
fairness  in  the  prediction  of  job  performance  for  minorities  in 
several  MOS.  In  addition,  a  change  in  the  computation  of  the 
Mechanical  Maintenance  (MM)  composite  and  recommended  changes  in 
required  aptitude  areas  for  approximately  50  MOS  are  scheduled 
for  implementation  in  the  near  future.  Annual  savings  from  the 
changes  in  composites  have  been  estimated  at  $25  million. 

The  Enlisted  Personnel  Allocation  System  (EPAS)  is  a  new, 
computerized  assignment  system  that  will  more  efficiently  match 
applicants  to  jobs  for  which  they  are  best  qualified,  maximizing 
performance  and  minimizing  attrition.  Thus,  with  significant 
improvements  in  classification  composites  (from  Project  A)  and  in 


assignment  (from  EPAS)  the  Army  can  put  the  "right"  person  in  the 
"riaht"  iob  at  the  "right  time.  The  potential  annual  sayings  to 
the  Army  from  using  ASVAB  (including  the  improved  composites)  in 
the  Enlisted  Personnel  Allocation  System  (EPAS)  to  predict 
performance  is  estimated  to  be  more  than  $480  million  annually. 

Results  from  the  Project  A  CV  have  demonstrated  conclusi¬ 
vely  that  ASVAB  is  an  excellent  test  of  the  "can  do"  or  more 
technical  task-based  requirements  of  Army  :)obs.  The  mean  validity 
coefficient  between  scores  on  ASVAB  and  Core  Technical  perfor¬ 
mance  is  .57  across  the  19  MOS  in  the  Concurrent  Validation. 

Since  ASVAB  was  designed  as  a  general  cognitive  test,  it  is  not 
expected  to  be  a  good  measure  of  more  specialized  spatial 
abilities,  psychomotor  skills,  motivation,  interests  or  leader¬ 
ship.  AS  a  means  of  supplementing  ASVAB,  the  new  Project  A  tests 
measure  these  characteristics  very  well. 


Spatial.  Psvchomotor  and  Perceptual  Tests.  Currently ,  ARI 
is  supporting  implementation  of  Project  A* s  spatial  and  com¬ 
puterized  psychomotor  and  perceptual  tests  in  USAREUR  and  at 
training  posts  in  CONUS. 


These  implementations  were  inspired  by  ARI  selection 
research  on  tank  and  anti-tank  gunners.  In  1986,  Project  A  s  new 
tests  had  a  multiple  R  of  .76  against  accuracy  on  high-tech 
simulators  of  tank  gunnery  for  95  Armor  officers  at  Fort  Knox. 
Cognitive  ability,  which  was  tested  as  well,  did  not  contribute 
to  the  prediction.  In  1987,  300+  new  recruits  at  Fort  Benning 
took  the  same  battery  before  training  in  anti-tank  gunnery. 
Several  of  the  new  tests  strongly  predicted  accuracy  in  firing 
the  TOW  (Tube-launched,  Optically-tracked,  Wire-guided)  missile 
simulator. 


Based  on  these  results,  in  December  87,  CG  TRADOC  ordered 
implementation  of  Project  A  tests  at  four  posts  that  train  on 
high-tech  weapons  systems.  ARI  installed  1-  and  2-hand  tracking, 
maze,  and  mental  rotation  tests  at  Forts  Knox,  Benning,  and  Bliss 
in  February  88. 

At  the  Infantry  and  Armor  sites,  the  earlier  results  have 
been  confirmed.  For  1,065  TOW  students  to  date,  41%  of  those 
passing  a  cut  score  on  the  predictors  qualified  as  gunners  in  the 
minimum  time,  as  opposed  to  24%  of  those  not  tested  on  the 
psychomotor/spatial  predictors  and  12%  of  those  scoring  below  the 
cut  score.  Those  scoring  above  the  cut  score  qualified  higher  as 
well  as  faster:  48%  attained  the  upper  two  levels  of  accuracy  as 
contrasted  with  36%  of  those  not  tested  and  31%  of  those  below 
1:he  cut  score#  In  terms  of  validities^  accuracy  was  predicted  at 
.37  by  the  Project  A  tests,  .29  by  ASVAB  GT,  and  .38  by  the  two 
together.  In  mid-88,  these  training  data  were  confirmed  in  live 
fire  at  Fort  Benning:  for  60  students  firing  one  live  TOW  each 
at  a  moving  target  at  6,000',  P(hit)  was  85%  for  those  who  met 
the  cut  on  the  Project  A  tests  and  73%  for  those  who  did  not. 


At  Fort  Knox,  Armor  recruits  (N=500+)  scoring  in  the  upper 
third  on  the  new  predictors  early  in  training  later  had  gunnery 
hit  rates  16%  higher  than  those  in  the  lower  third,  exactly 
repeating  the  results  in  the  1986  research.  The  new  predictors 
correlated  .54  with  speed/accuracy,  compared  with  a  .34  validity 
for  the  ASVAB  GT  composite.  Combined,  the  validity  was  .55, 
reconfirming  the  importance  of  spatial  and  psychomotor  skills  in 
predicting  gunnery  performance.  At  both  posts,  the  new  results 
were  so  positive  that  the  test  scores  were  incorporated  into  the 
decision  making  process. 

Validities  at  Fort  Bliss  were  not  significant,  and  testing 
there  has  been  suspended.  In  contrast  to  tank  and  anti-tank 
weapons,  which  rely  heavily  on  tracking,  the  air  defense  systems 
tested  use  fire-and-forget  weapons  against  fleeting  targets. 

Thus  these  systems  rely  more  on  skills  like  vigilance  and  target 
identification. 

Completing  the  implementation  requests  from  CG  TRADOC,  the 
Field  Artillery  School,  Fort  Sill,  is  starting  to  administer  a 
broader  battery  of  the  new  tests  to  recruits  in  meteorology, 
surveying,  radar  range-finding,  and  artillery  spotting  (MOS) . 
Validation  will  be  against  performance  at  the  end  of  training. 

Currently,  the  active  forces  are  transitioning  to  the 
Bradley  Fighting  Vehicle.  In  USAREUR,  V  Corps  has  elected  to 
use  a  battery  of  spatial  and  psychomotor  tests  to  inform  their 
selection  of  Bradley  gunners  in  the  3rd  Armor  Division  and  the 
8th  Infantry  Division. 

A  need  for  improved  selection  has  been  found  also  at  the 
Special  Forces  School,  where  attrition,  due  primarily  to 
failures  in  land  navigation,  is  costly.  In  a  new  pilot  project, 
ARI  has  installed  three  spatial  tests. at  Fort  Bragg  to  identify 
good  land  navigators. 

In  review,  early  positive  results  are  being  replicated  in 
the  implementations;  spatial/psychomotor  abilities  strongly 
predict  differences  in  gunnery  performance;  and  utilization  is 
spreading.  Budget  reductions,  however,  threaten  the  survival  of 
the  implementation  in  the  training  base.  In  the  future,  the 
tests  could  have  the  greatest  impact  if  they  were  administered 
before  enlistment.  In  that  case  the  Army  Reserve  and  National 
Guard  could  be  served  as  well,  and  initial  person- job  matching 
could  be  strengthened  in  many  MOS  by  adding  spatial  and/pr 
psychomotor  abilities  to  the  profile  of  aptitudes. 

Assessment  of  Background  and  Life  Experiences  (ABI^l,.  The 
ABLE  is  a  30-minute,  multiple  choice,  non-cognitive  test  designed 
to  measure  temperament,  personal  history,  and  adaptability.  In 
the  Project  A  research,  ABLE  was  shown  to  improve  significantly 
the  prediction  of  the  motivational  components  of  performance. 
Scores  on  the  ABLE  are  strongly  related  to  12-month  attrition. 
Recruits  who  have  very  low  scores  on  the  ABLE  Adjustment 


composite  have  attrition  rates  that  are  two  to  three  times  higher 
than  soldiers  with  high  Adjustment  scores. 

The  Dependability  scale  of  ABLE  predicts 
nary  problems.  For  example,  soldiers  having  low  Dependability 
ecwes  receive  significantly  more  Articles  15  than  those  with 
hiah  sco?Sr  cln?ersely,  soldiers  with  high  Dependability  scores 
are  more  often  viewed  as  having  potential  of  becoming  high 
performing  NCOS. 

In  sum,  the  ABLE  shows  promise  to  augment  toe  Services' 

capabiu!S  t“identify  the  Sost  f 

categories  and  educational  levels  by  predicting  toe^ 
components  of  performance.  Based  ^  ' 

the  ABLE  is  to  be  included  in  a  goint  Service  adaptability 
screening  instrument  that  will  become  operational  in  1990. 

eerheI^"..°srlnduringpr^SIt^ggS%hyfroj^t  ^?^e4r={;^ffort 

lomes  from  the  Longitudinal  Research  ® 

is  a  permanent  storehouse  of  empirical  information  on  Prefect  a, 
unparalleled  in  its  richness  for  addressing  recurring  Army  con 
cerns.  These  data  are  invaluable  to  the  Army  in  the  areas  of 
accession  policy,  setting  standards  for  enlistment  and  reenlist- 
ment,  predicting  attrition,  linking  school  training  to 
performance,  and  linking  characteristics  at  entry  J-Jto  toe 
military  to  performance  in  first  and  second  tours  of  duty. 

project  A  was  initiated  in  1982  and  will 
1989.  While  Project  A  has  focused  primarily  on  Aray  enlisted 
personnel  in  their  first  tour  of  service,  a  second  res epch 
program?  "BSilding  and  Retaining  the  Career  Force,”  will  foc^s  on 
the  Army's  career  force,  those  soldiers  who  reenlist  and  remain 
beyond  one  tour  of  duty.  Soldiers  who  comprise  the  career  force 
provide  not  only  leadership  and  continuity  to  toe  Army  as  an  - 
institution,  but  are  crucial  in  insuring  that  the  Army  meets  its 
battlefield  mission. 
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INTRODUCTION 


Since  1979  military  testing  has  been  the  focus  of  increasing 
attention,  from  both  the  Department  of  Defense  and  Congress.  The 
reasons  for  this  interest  are  varied  and  include: 

1.  The  discovery  in  1979  that  the  then  operational 
selection  test  battery  for  military  enlistment 
was  misnoraed. 

2.  The  switch  in  reference  groups  for  military 
tests  from  the  1944  mobilization  population 
(all  men,  including  officers,  serving  under 
arms  during  that  year)  to  the  1980  youth 
population  (all  17-  to  23-year  olds  in  1980 
including  both  males  and  females) . 

3.  A  Congressional  mandate  to  link  entry  test 
scores  directly  to  on-the-job  performance 
rather  than  to  traditional  training 
performance. 

As  an  integrated  response  to  these  various  concerns  the  U.S. 
Army  Research  Institute  designed  a  large-scale  research  program 
to  improve  the  Army's  personnel  system.  The  first  phase  of  this 
program.  Project  A,  focused  primarily  on  initial  selection  and 
classification.  Its  objectives  included:  (1)  validation  of  the 
existing  enlisted  selection  and  classification  battery,  the  Armed 
Services  Vocational  Aptitude  Battery  (ASVAB) ,  (2)  development  and 
validation  of  new  cognitive  and  non-cognitive  measures  that  add 
to  the  ASVAB  in  predicting  total  soldier  performance,  and  (3) 
development  of  a  comprehensive  set  of  performance  measures 
including  paper-and-pencil  tests  of  school  knowledge  and  job 
knowledge,  hands-on  tests  of  job  performance,  and  behavioral ly- 
anchored  rating  scales. 

Nineteen  Military  Occupational  Specialties  (MOS)  were 
selected  as  a  representative  sample  of  the  Army's  250+ 
entry-level  jobs.  Nine  of  the  19  MOS  were  then  selected  to  have 
both  paper-and-pencil  and  hands-on  tests  of  job  performance  in 
addition  to  the  school  knowledge  tests  and  ratings  administered 
in  all  of  the  MOS.  The  nine  fully-tested  MOS  in  Project  A  are: 
Infantryman  (IIB),  Cannon  Crewman  (13B) ,  Tank  Crewman  (19E/K) , 
Radio  Operator  (31C),  Light  Wheel  Vehicle  Mechanic  (63B) , 
Administrative  Specialist  (71L) ,  Motor  Transport  Operator  (88M) , 
Medical  Care  Specialist  (91A) ,  and  Military  Police  (95B) . 

The  first  stage  of  the  Project  A  research,  the  Concurrent 
Validation  (CV) ,  was  conducted  in  FY  1985.  In  the  CV,  over  9400 
soldiers  in  the  19  MOS  were  administered  the  new  Project  A 
predictor  tests,  which  included  measures  of  spatial  abilities, 
temperament  and  vocational  interest,  as  well  as  computerized 
tests  of  perceptual  and  psychomotor  skills.  These  tests  were 


designed  to  expand  the  predictor  domain  in  terms  of  individual 
characteristics  and  attributes  that  might  be  important  for 
selection  in  predicting  more  aspects  of  performance.  The  new 
predictors  were  intended  to  supplement  those  cognitive  abilities 
assessed  by  ASVAB. 

Concurrently,  the  CV  soldiers,  who  had  been  in  the  Army  12 
to  27  months,  were  administered  a  set  of  job  performance  measures 
including  supervisory  and  peer  ratings,  written  school  and  job 
knowledge  tests,  and  MOS-specific  hands-on  task  proficiency 
measures. 

One  of  the  major  contributions  of  Project  A  to  date  is  a 
comprehensive  description  of  the  major  components  of  job  perfor¬ 
mance  for  the  first  tour  soldier.  Many  measurement  methods  were 
used  in  this  project  to  assess  all  aspects  of  performance. 
Analyses  of  the  data  collected  during  the  CV  resulted  in  the 
identification  of  five  performance  dimensions,  as  follows;  The 
five  Project  A  job  performance  dimensions  are: 

1.  MOS-Technical  Knowledge  and  Skill.  The 
proficiency  with  which  a  soldier  performs  the 
technical  tasks  that  are  critical  and  central 
to  his/her  MOS. 

2.  General  Soldiering  Proficiency;  How  well  a 
soldier  executes  common  soldiering  tasks  such 
as  first  aid  and  land  navigation. 

3.  Effort  and  Leadership;  The  degree  to  which  a 
soldier  exerts  effort  over  the  full  range  of  job 
tasks,  perseveres  under  adverse  or  dangerous 
conditions,  and  demonstrates  leadership  and  support 
toward  peers. 

4.  Personal  Discipline;  The  degree  to  which  a 
soldier  adheres  to  Army  regulations,  exhibits 
self-control  and  does  not  create  disciplinary 
problems . 

5 .  Physical  Fitness  and  Military  Bearing;  The 
degree  to  which  a  soldier  maintains  an  appro¬ 
priate  military  appearance  and  remains  in  good 
physical  condition. 

The  first  two  performance  dimensions  involve  a  soldier's 
ability  to  perform  the  technical  requirements  of  his/her  job  and 
are  often  referred  to  as  "can  do"  factors.  The  remaining  three 
performance  dimensions  are  more  attitudinal  or  motivational  in 
nature  and  are  often  called  "will  do"  factors. 
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The  second  stage  of  the  Project  A  research,  known  as  the 
Longitudinal  Validation  (LV) ,  started  in  1986  with  the  admin¬ 
istration  of  the  battery  of  new  Project  A  predictor  tests  to 
approximately  50,000  new  recruits  in  21  MOS.  At  the  end  of 
his/her  Advanced  Individual  Training  (AIT)  or  One  Station  Unit 
Training  (OSUT) ,  each  soldier  was  administered  the  school 
knowledge  test  for  his/her  MOS,  and  a  set  of  rating  scales  was 
collected  from  supervisors  and  peers.  Approximately  10,000  of 
these  soldiers  were  followed  into  their  first-tour  field 
assignment  and  were  administered  a  set  of  MOS-specific  and 
Army-wide  job  performance  measures  during  FY88  and  FY89.  In 
addition,  about  1,000  soldiers  were  assessed  on  a  preliminary 
battery  of  performance  measures  appropriate  for  second-tour 
including  rating  scales,  written  job  knowledge  tests,  hands-on 
measures,  and  leadership  measures. 


Results  and  Products 


Project  A  delivered  four  major  products  to  the  Army. 

ASVAB .  One  product  which  resulted  from  the  Concurrent 
Validation  research  effort  involves  improvements  in  the  computa¬ 
tion  and  use  of  ASVAB  Aptitude  Area  (AA)  Composites.  These 
composites  are  used  to  determine  eligibility  for  training  in  Army 
jobs.  Beginning  with  the  operational  implementation  of  ASVAB 
11/12/13/14  in  October  1984  the  formulae  for  the  Clerical  (CL) 
and  the  Surveillance/Communications  (SC)  composites  were  changed. 
These  changes  resulted  in  both  better  accuracy  and  improved 
fairness  in  the  prediction  of  job  performance  for  minorities  in 
several  MOS.  In  addition,  a  change  in  the  computation  of  the 
Mechanical  Maintenance  (MM)  composite  and  recoinmended  changes  in 
required  aptitude  areas  for  approximately  50  MOS  are  scheduled 
for  implementation  in  the  near  future.  Annual  savings  in  reduced 
attrition  and  improved  job  performance  from  the  changes  in 
composites  have  been  estimated  at  $25  million. 

Results  from  the  Project  A  CV  have  demonstrated  conclusively 
that  ASVAB  is  an  excellent  test  of  the  "can  do"  or  more  technical 
task-based  requirements  of  Army  jobs.  The  mean  validity 
coefficient  between  scores  on  ASVAB  and  Core  Technical  perfor¬ 
mance  is  .57  across  the  19  MOS  in  the  Concurrent  Validation. 

Since  ASVAB  was  designed  as  a  general  cognitive  test,  it  is  not 
expected  to  be  a  good  measure  of  more  specialized  spatial  abilit¬ 
ies,  psychomotor  skills,  motivation,  interests  or  leadership.  As 
a  means  of  supplementing  ASVAB,  the  new  Project  A  tests  measure 
these  characteristics  very  well. 

Spatial.  Psvchomotor  and  Perceptual  Tests.  For  success  in 
tube- launched,  optically-tracked,  wire-guided  (TOW)  missile  and 
tank  gunnery,  two  types  of  special  skills  are  required:  one  is 
the  ability  to  deal  with  the  orientation,  location,  and  shape  of 
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objects  (spatial  ability) ;  the  other  is  eye-hand  coordination 
(psychomotor  ability) . 

Project  A  developed  new  tests  of  these  special  skills. 
Spatial  abilities  are  tested  on  paper  with  mazes  and  mental 
rotation  items  (where  the  soldier  imagines  how  something  will 
look  after  it  has  been  turned) .  Psychomotor  ability  is  tested  on 
a  device  resembling  a  video  game.  Using  hand  controls,  a  soldier 
tries  to  keep  a  crosshair  on  a  moving  target. 

Late  in  1987,  GEN  Maxwell  R.  Thurman,  then  Commanding 
General  of  TRADOC,  reviewed  the  results  of  preliminary  research 
on  these  tests  and  directed  the  tests  be  installed  at  selected 
posts  where  gunnery  skills  are  trained.  From  February  1988 
through  August  1989,  Forts  Banning,  Knox,  and  Bliss  tested  over 
24,000  new  soldiers.  At  the  end  of  training,  many  of  these 
soldiers  were  tested  on  live  fire  or  simulator  gunnery  perfor¬ 
mance.  Students  in  the  IIH  TOW  Gunner  course  who  had  high 
gunnery  aptitude  (i.e.,  upper  40  percentiles)  on  the  Project  A 
tests  qualified  in  fewer  trials  and  were  more  accurate  than  those 
with  low  gunnery  aptitude.  For  example,  the  high  aptitude 
students  had  a  hit  rate  in  live-fire  that  was  11%  higher  than  the 
other  students.  These  findings  were  replicated  on  TOW  gunners  in 
USAREUR  in  1989,  the  hit  rates  in  live  fire  being  92%  and  60%  for 
high  and  low  aptitude  soldiers,  respectively.  Similar  results 
emerged  at  Fort  Knox:  19K  Armor  students  with  high  gunnery 
aptitudes  had  hit  rates  7%  higher  than  the  average,  and  they 
acquired  targets  faster.  Currently,  USAREUR  units  are  evaluating 
the  usefulness  of  these  Project  A  tests  for  the  selection  of 
Bradley  gunners. 

Thus,  spatial  and  psychomotor  abilities  have  been  found  to 
strongly  predict  differences  in  gunnery  performance.  Currently, 
a  battery  of  Project  A  tests  is  being  administered  at  Forts 
Benning,  Knox,  and  Sill  as  a  step  toward  possible  full-scale  im¬ 
plementation  in  the  Joint  Service  selection  testing  process. 

Assessment  of  Background  and  Life  Experiences  TABLED .  The 
ABLE  is  a  30-minute,  multiple  choice,  non-cognitive  test  designed 
to  measure  temperament,  personal  history,  and  adaptability.  In 
the  Project  A  research,  ABLE  was  shown  to  improve  significantly 
the  prediction  of  the  motivational  components  of  performance. 
Scores  on  the  ABLE  are  strongly  related  to  12 -month  attrition. 
Recruits  who  have  very  low  scores  on  the  ABLE  Adjustment  com¬ 
posite  have  attrition  rates  that  are  two  to  three  times  higher 
than  soldiers  with  high  Adjustment  scores. 

The  Dependability  scale  of  ABLE  predicts  in-service 
disciplinary  problems.  For  example,  soldiers  having  low 
Dependability  scores  receive  significantly  more  Articles  15  than 
those  with  high  scores.  Moreover,  soldiers  with  high  Depen- 
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dability  scores  are  more  often  viewed  as  having  potential  of 
becoming  high  performing  NCOS. 

In  sum,  the  ABLE  shows  promise  to  augment  the  Sewices' 
capability  to  identify  the  most  qualified  applicants  in  all  AFQT 
categories  and  educational  levels  by  predicting  the  "will  do” 
components  of  performance.  Because  of  the  encouraging  results 
obtained  thus  far,  the  ABLE  is  now  being  examined  as  a  potential 
tool  in  the  selection  process  for  both  the  Rangers  and  Special 
Forces.  It  has  also  incorporated  into  a  joint  Service  adap¬ 
tability  screening  instrument  that  is  being  considered  for 
initial  operational  testing  in  1991. 

The  Longitudinal  Research  Data  Base.  The  fourth,  and 
perhaps  most  enduring  product  from  the  Project  A  research  effort 
comes  from  the  Longitudinal  Research  Data  Base  (LRDB) .  The  LRDB 
is  a  permanent  storehouse  of  Project  A  data,  unparalleled  in  its 
richness  for  addressing  recurring  Army  concerns.  These  data  are 
invaluable  to  the  Army  in  the  areas  of  accession  policy,  setting 
standards  for  enlistment  and  reenlistment,  predicting  attrition, 
linking  school  training  to  field  performance,  and  linking  charac¬ 
teristics  at  entry  into  the  military  to  performance  in  the  first 
tour  of  duty. 


Follow-on  Research  to  Project  A 

Project  A  was  initiated  in  1982  and  was  completed  in  March, 
1990.  By  demonstrating  that  the  principal  current  selection  and 
classification  tool,  the  ASVAB,  was  doing  an  effective  job  of 
predicting  performance,  it  provided  justification  for  continued 
use  of  this  valuable  tool.  Further,  it  generated  new  tests  and 
demonstrated  that  substantial  improvements  could  be  made  to  the 
current  system  by  adding  these  tests. 

What  remains  is  to  use  the  information  collected  during  the 
Longitudinal  Validation  of  Project  A  to  determine  how  best  to 
combine  old  and  new  tests  in  a  revamped,  improved,  initial  entry 
screening  process.  This  is  one  of  the  objectives  of  Building  the 
Career  Force,  the  second  phase  of  the  program  to  improve  the 
Army's  personnel  system.  A  second  objective  of  this  follow-on 
project  will  be  to  provide  a  foundation  for  improved  reenlistment 
and  promotion  decisions.  When  Building  the  Career  Force  is 
complete,  the  Army  will  have  the  basis  for  building  a  system 
which  integrates  personnel  actions  from  a  soldier's  initial  entry 
into  the  Army  through  promotion  to  the  rank  of  sergeant,  with  all 
actions  guided  toward  the  goal  of  maintaining  maximum  readiness 
and  fighting  effectiveness  of  the  entire  Army  organization. 
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A  SUMMARY  OF  THE  DEVELOPMENT  PROCEDURES  FOR 
JOB  PERFORMANCE  MEASUREMENT  IN  PROJECT  A 

This  paper  summarizes  procedures  being  used  in  Project  A  to  develop 
hands-on  performance  measures  for  Army  job  specialties.  Job  and  task 
analysis  procedures  are  covered  in  four  phases:  defining  the  task  domain, 
identifying  task  characteristics,  selecting  tasks  for  testing,  and  detail¬ 
ing  task  procedures.  A  section  on  criterion  test  development  is  also 

i ncl uded . 

Since  job  specific  performance  tests  are  to  be  tied  to  job  tasks,  job 
performance  must  first  be  broken  down  into  tasks,  a  sample  of  those  tasks 
selected,  then  each  further  analyzed  into  its  elemental  parts.  While  a 
task  inventory  approach  to  job  analysis  has  some  disadvantages  in  that 
important  contextual  factors  and  translation  behaviors  can  be  lost  in  the 
"seams"  when  a  job  is  partitioned  into  tasks,  the  approach  has  a  number  of 
important  advantages.  Chief  among  them  is  the  exceptional  degree  of  con¬ 
trol  and  standardization  that  can  be  achieved  in  testing  performance  task 
by  task.  Another  is  analytic  convenience:  judgments  about  mission  rele¬ 
vance,  frequency  of  performance,  kinds  of  behavior,  and  other  data 
important  to  achieving  content  validity  can  be  obtained  systematically  and 
reliably  on  a  task  by  task  basis.  A  final  advantage  particular  to  this 
project  is  that  the  Army's  occupational  analysis  system  is  based  on  a  task 
inventory  approach. 

Defining  the  Task  Doipain 

The  first  stage  of  job  analysis  entails  identifying  the  population  of 
entry-level  (SIjill  Level  1)  tasks  for  an  Army  job  specialty  (MOS). 


The  Army  Occupational  Survey  Center  (AOSC),  working  with  the  technical 
school  responsible  for  job  and  task  analysis,  routinely  develops  task  lists 
for  the  various  job  specialties.  A  survey  form  based  on  the  task  list  is 
used  to  obtain  data  from  job  incumbents  on,  among  other  things,  tasks  per¬ 
formed  and  frequency  of  performance.  The  task  content  that  is  prescribed 
for  this  job  by  Army  management  is  listed  and  described  in  some  detail  in 
the  Soldier's  Manual  (SM)  for  an  MOS.  The  Soldier's  Manual  descriptions 
and  related  survey  data  encompass  the  range  of  doctrine  (SM)  and  practice 
(AOS)  in  job  performance.  The  two  sources  of  task  data  differ  also  in  the 
level  of  task  statement.  The  Soldier's  Manual  describes  job  content  at  a 
somewhat  more  general  level  than  do  the  survey  items.  To  consolidate  the 
two  sources,  a  five  step  procedure  is  followed. 

The  first  step  is  to  use  the  occupational  survey  data  to  identify  AOS 
tasks  or  activities  that  are  reported  as  actually  performed  by  Skill  Level 
1  (SLl)  incumbents.  This  is  a  matter  of  screening  the  frequency-of- 
performance  data  for  all  AOS  activities  to  which  SLl  incumbents  responded, 
and  retaining  those  with  non-zero  frequencies. 

The  next  step  is  to  map  the  SLl  survey  items  onto  tasks  prescribed  in 
the  SM.  An  AOS  activity  is  placed  under  an  SM  task  if  the  statement  dupli¬ 
cates  the  SM  task  or  is  subsumed  under  the  SM  task  as  a  step  or  variation 
in  conditions.  The  intent  is  to  identify  SLl  SM  tasks  with  which  the  AOS 
activity  can  be  matched.  If  this  can  not  be  done,  higher  skill  level  tasks 
described  in  the  SM  are  successively  reviewed  and  the  AOS  questionnaire 
items  matched  where  possible  with  those  SM  tasks.  The  objective  is  to 
match  AOS  statements  of  practice  with  the  SM  statements  of  doctrine  where 
possible,  even  if  doctrine  does  not  specifically  identify  the  activity  as  a 
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SLl  responsibility.  All  SLl  SM  tasks  are  retained  regardless  of  whether  or 
not  they  had  parallel  AOS  statements. 

Since  some  AOS  questionnaire  items  can  not  be  matched  with  any  SM 
task»  or  any  subset  of  elements  from  an  SM  task,  the  third  step  is  to  edit 
the  remaining  AOS  task  statements  (i.e.,  items)  so  that,  while  similar  in 
format  to  the  SM  task  statements,  they  are  still  a  clear  portrayal  of  addi¬ 
tional  task  content  not  contained  in  the  SM.  In  some  cases  an  AOS  state¬ 
ment  becomes  a  task  statement  by  itself;  in  other  cases  a  new  task  state¬ 
ment  is  developed  which  appropriately  subsumes  several  AOS  statements.  The 
objective  is  to  write  all  task  descriptions  at  the  same  level  of 
generality. 

The  final  step  is  to  have  the  consolidated  list  of  task  statements 
reviewed  by  subject  matter  experts  (SME)  at  the  proponent  school.  Usually 
three  senior  NCO's  review  the  list  for  an  MOS.  The  review  focuses  on  the 
correctness  of  the  matching  of  survey  items  with  SM  task  descriptions,  the 
appropriateness  of  the  task  titles  for  the  AOS-only  tasks,  and  relevance  of 
tasks  in  light  of  the  latest  changes  in  equipment  or  doctrine.  The  con¬ 
solidation  of  SM  and  AOS  results  in  an  MOS  task  domain  for  Skill  Level  1. 
The  domain  includes  all  SLl  tasks  from  the  Soldier's  Manual  for  a  parti¬ 
cular  MOS  and  their  supporting  AOS  statements,  all  higher  skill  level  tasks 
with  supporting  AOS  statements,  and  AOS-only  tasks.  Described  in  this  way, 
the  domain  portrays  in  precise  task  descriptive  terms  a  definition  of  the 
job-world  that  an  SLl  incumbent  will  face. 

But  even  with  consolidation  the  domain  typically  tends  to  span  200 
to  300  tasks  which  is  too  many  to  be  managed  feasibly  by  SMEs  from  whom 
ratings  of  task  criticality,  difficulty  and  similarity  are  to  be  obtained. 
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So  the  domain  is  subjected  to  further  consolidation.  First,  highly  similar 
tasks  that  differ  only  because  of  equipment  variations  are  consolidated. 
Next,  tasks  are  deleted  that  pertain  to  peripheral  or  restricted  duty  posi¬ 
tions  in  the  MOS.  Also  deleted  are  higher  skill  level  tasks  doctrinally 
designated  as  irrelevant  to  SLl  job  performance.  Then  any  AOS-only  (i.e., 
non-doctrinal  survey  derived)  tasks  with  atypically  low  frequency-of- 
performance  indexes  are  dropped.  Finally,  any  existing  data  on  task  criti¬ 
cality  may  be  used  to  eliminate  tasks  judged  by  job  experts  to  be 
unimportant. 

Application  of  these  final  filters  tends  to  narrow  the  domain  to 
approximately  150  tasks. 

Identifying  Task  Characteristics 

All  tasks  in  the  domain  are  now  viewed  as  candidates  for  testing. 
Since  the  objective  is  to  select  from  the  domain  a  sample  of  tasks  that  is 
content  valid,  it  is  necessary  at  this  stage  to  collect  data  on  task  char¬ 
acteristics  pertinent  to  content  validity.  For  Project  A  these  have  been 
designated  as  task  importance,  difficulty,  and  similarity.  Data  on  fre¬ 
quency  of  performance  is,  as  mentioned,  available  from  AOS. 

Using  the  rating  protocols  shown  as  Attachment  1,  data  on  these  task 
characteristics  are  obtained  from  15  job  experts  (SMEs)  at  the  proponent 
school.  For  each  set  of  ratings— importance,  similarity  and  difficulty— an 
SME  works  with  a  deck  of  task  cards,  each  card  listing  the  title  of  a  task 
and  a  brief  description  of  the  task  scope.  In  separate  sessions,  SMEs  pro¬ 
vide  the  three  types  of  judgments:  each  (1)  ranks  the  tasks  as  to  impor¬ 
tance  in  carrying  out  a  prescribed  unit  mission,  (2)  sorts  the  tasks  into 
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groups  on  the  basis  of  similar  performance  requirements,  and  (3)  estimates 
the  typical  distribution  of  soldiers  over  five  categories  of  task 


proficiency. 

The  importance  index  is  the  rank  of  the  task  averaged  over  judges. 
Similarity  is  indexed  two  ways:  one  is  the  number  of  judges  who  placed  the 
task  with  every  other  task;  the  other  is  cluster  membership,  that  is  the 
cluster  containing  the  task,  which  is  identified  through  a  factor  analysis 
of  the  cross-products  matrix  derived  from  the  judges'  task  similarity 
clusters. 

These  task  variables,  together  with  frequency,  are  used  in  the  next 
stage  of  the  analysis,  that  of  task  selection— the  goal  being  to  select  for 
testing  tasks  that  are  important,  frequently  performed,  of  moderate  diffi¬ 
culty,  and  that  represent  a  range  of  content. 

Task  Selection 

Data  from  SME  ratings  on  task  importance,  difficulty  and  similarity 
are  used  along  with  the  occupational  survey  (AOS)  data  on  frequency  of 
performance  to  select  30  tasks  for  testing.  That  number  was  judged  to 
approach  the  outside  limit  that  could  be  handled  by  the  resources  of  the 
project.  Job  analysts  on  the  project  staff  process  the  data  following  a 
four  phase  procedure. 

In  the  first  phase,  analysts  are  given  the  task  data  derived  from  the 
job  experts'  ratings  and  asked  to  select  30  tasks  to  represent  each  MOS,  so 
as  to  maximize  the  content  validity  of  the  task  sample.  No  strict  rules 
are  provided  to  the  analysts  for  making  their  selections. 
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The  second  phase  employes  the  use  of  policy  capturing  to  identify  each 
analyst's  general  tendency  in  using  the  data  they  were  provided.  Then, 
each  analyst's  task  selections  for  each  MOS  are  regressed  on  the  task  char¬ 
acteristics  data.  The  resulting  regression  equations  identify  the  implicit 
policies  used  in  selecting  tasks.  The  equations  are  then  applied  to  the 
task  characteristics  data  to  provide  an  estimate  of  the  task  selections 
each  would  have  made  if  their  selections  were  completely  consistent  with 
their  general  tendency,  as  represented  by  a  linear  model. 

In  the  third  phase,  analysts  are  given  their  original  task  selections 
and  the  selections  predicted  by  their  regression-captured  policies.  Dif¬ 
ferences  may  be  expected  between  the  two  lists  of  tasks.  Some  deviations 
are  expected  because  an  analyst  may  have  used  something  other  than  a  linear 
combination  of  the  task  data  or  may  have  used  knowledge  about  the  MOS  tasks 
not  represented  in  the  task  characteristics  data  provided.  On  the  other 
hand,  some  deviations  may  be  due  simply  to  unsystematic  treatment  of  the 
data.  Therefore,  the  purpose  of  phase  three  is  to  have  the  analysts  review 
and  justify  discrepancies  between  their  observed  and  predicted  selections. 
Discrepancies  that  resulted  from  unsystematic  treatment  of  the  data  can  be 
corrected,  and  the  rationale  for  each  intentional  discrepancy  can  be  ident¬ 
ified. 

The  final  phase  of  the  procedure  is  a  Del  phi- type  negotiation  among 
analysts  to  converge  their  respective  choices  onto  30  tasks  for  each  MOS. 
The  choices  and  rationale  produced  in  phase  three  for  each  analyst  are  for¬ 
matted  and  distributed  to  all  analysts.  Analysts  can  then  retain  or  adjust 
their  selections  based  on  the  opinion  of  others.  The  revisions  are  col¬ 
lected,  formatted  and  redistributed  for  the  analysts  to  make  further 
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revisions.  This  process  is  repeated  until  consensus  is  reached  on  30  tasks 
for  each  MOS. 

The  sample  of  tasks  selected  for  testing  by  the  measurement  special¬ 
ists  is  then  reviewed  by  a  panel  of  job  experts  from  the  proponent  school 
for  relevance,  completeness,  and  fairness. 

In  the  final  stage  of  task  description  for  test  development,  detailed 
descriptions  of  the  steps  and  procedures  in  task  execution  are  assembled 
for  each  selected  task.  Existing  descriptive  data  from  Field  Manuals, 
Technical  Manuals,  Soldier's  Manuals,  school  lesson  plans,  and  other  task 
analysis  research  are  used  in  detailing  the  procedures,  conditions  and 
standards  for  performing  a  task  (see  examples  in  Attachment  2).  Accuracy 
and  completeness  of  the  descriptions  are  confirmed  through  additional  job 
expert  reviews.  The  purpose  for  detailing  these  aspects  of  task  perfor¬ 
mance  is  to  provide  the  "items"  that  will  be  scored  either  in  a  paper-and- 
pencil  or  hands-on  testing  mode. 

To  bridge  the  gap  between  the  assembled  task  detailing  and  hands-on 
performance  measures  it  is  necessary  to  determine  the  test- task 
boundaries.  To  do  this,  the  developer  determines  the  administrative 
limitations  at  the  test  site.  The  primary  limitation  is  usually  time; 
however,  equipment  availability,  environmental  limitations,  terrain, 
support  troops  and  potential  equipment  and  personal  risks  also  affect  how 
much  of  the  task  can  be  tested.  These  factors  often  dictate  that  the  task 
be  reconfigured  into  one  that  meets  test  feasibility  requirements  and  the 
task  description  is  modified  accordingly.  But,  before  test  development 
begins,  the  developer  must  make  a  final  check  to  confirm  that  the  modified, 
feasible  task  still  reflects  the  task  characteristics  that  dictated  its 
inclusion  originally. 
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with  the  assembling  of  the  task  analytic  material  and  a  rough  outline 
of  the  hands-on  scope  of  the  task,  test  development  begins. 

Selection  of  Test  Mode 

The  composite  MOS  test  to  be  administered  to  cohorts  will  consist  of  a 
mix  of  hands-on  and  knowledge  measures,  each  used  for  the  type  of  task  to 
which  it  is  best  suited.  If  knowledge  tests  correlated  high  enough  with 
hands-on  tests  across  the  range  of  task  types,  we  could  rely  on  knowledge 
tests  alone,  saving  substantially  in  administrative  time  and  resources. 
But  at  this  stage  we  can  only  speculate  about  the  kinds  of  tasks  that  can 
and  cannot  be  tested  validly  in  a  knowledge  mode.  In  Project  A  our  intent 
is  to  examine  through  a  field  test  the  knowledge-hands-on  relationship  for 
a  variety  of  tasks,  deciding  on  the  basis  of  the  correlational  evidence 
those  tasks  for  which  a  knowledge  test  may  be  used  and  those  for  which  it 

may  not. 

So,  for  the  experimental  field  test,  both  kinds  of  tests  are  developed 
for  half  of  the  tasks  in  an  MOS.  (Hands-on  tests  for  15  tasks  is  all  that 
administration  time  will  support.)  The  characteristics  of  tasks  for  which 
hands-on  tests  are  expected  to  be  necessary  are:  (a)  psychomotor  skill, 
(b)  critical  time  limits  on  performance,  and  (c)  performance  steps  uncued 
by  task  sequence  or  task  conditions  (i.e,  steps  that  cannot  be  tested  in  a 
multiple-choice  knowledge  test  without  cueing  the  examinee).  A  rating 
procedure  focussing  on  these  three  task  characteristics  is  used  by  three 
task  analysts  to  evaluate  each  of  the  30  tasks.  Tasks  with  high  ratings  on 
all  three  dimensions  are  the  first  designated  for  hands-on  test  develop¬ 
ment,  since  we  suspect  these  will  be  most  likely  used  in  cohort  measure¬ 
ment.  The  hands-on  set  is  filled  out  with  tasks  representing  the  remaining 
range  of  scale  values. 
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Knowledge  tests  are  developed  for  all  30  tasks  in  the  sample. 


Test  Construction 

Development  of  a  hands-on  test  begins  with  the  task  descriptive  data 
and  proceeds  through  four  stages.  The  scoring  approach  is  determined 
first.  Here  a  decision  is  made  as  to  whether  test  scoring  will  be  directed 
at  the  product  or  the  process  of  task  performance,  or  both.  The  question 
to  be  answered  is,  what  must  the  scorer  observe  to  evaluate  a  soldier's 
performance?  If  task  performance  results  in  a  product,  then  scoring  can 
focus  on  the  measurable  characteristics  of  that  product.  However,  since 
the  relationship  of  task  process  to  task  outcome  is  not  always  evident, 
some  element  of  performance — safety  precautions,  for  instance — must  be 
observed  and  scored  as  they  occur. 

In  the  second  stage  of  development  these  product  and  process  charac¬ 
teristics  are  translated  into  a  score  sheet  or  list  of  observable  items  to 
be  evaluated  by  the  scorer.  It  should  be  noted  that  these  scoreable  items, 
or  what  we  term  performance  measures,  are  not  merely  a  transcription  of 
steps  from  the  task  description.  Some  task  elements  may  have  to  be  broken 
down  into  smaller  parts  if  scoring  is  to  be  done  reliably.  In  other  cases, 
a  step  may  be  eliminated  altogether  from  the  scoring  protocol  because  it  is 
neither  overtly  observable  nor  produces  an  observable  result.  Included  as 
performance  measures  are  steps  or  elements  molecular  enough  to  be  scored 
reliably  by  a  trained  evaluator  with  or  without  scoring  aids.  Dimensions 
of  overall  task  performance  such  as  time  and  sequence  are  added  to  the 
scoring  protocol  where  important. 
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The  third  stage  in  developing  a  hands-on  test  involves  preparing 
detailed  specifications  for  test  administration,  including  the  precise  con¬ 
ditions  (environment,  equipment,  terrain,  etc.)  for  testing,  instructions 
to  examinees,  and  instructions  to  scorers. 

Once  the  draft  test  package  has  again  been  reviewed  by  job  experts, 
development  moves  to  the  fourth  and  final  stage  in  which  through  a  small 
scale  tryout  the  test  is  evaluated  for  administrative  feasibility,  scorer 
reliability  and  acceptability  to  examinees.  Final  revisions  are  guided  by 
the  results  of  this  trial  run.  Sample  hands-on  tests  for  two  Artilleryman 
(MOS  13B)  tasks  are  given  in  Attachment  3. 

Development  of  the  knowledge  test  begins  with  the  same  task  descrip¬ 
tive  data.  But  here,  in  contrast  to  hands-on  measures  which  are  limited  to 
observable  task  elements,  covert  or  mental  steps  in  task  performance  can  be 
included.  Methodically  linking  test  questions  to  task  procedures  is  the 
key  to  valid  tests  of  job  knowledge.  And  the  sequence  of  decisions  and 
actions  to  be  followed  in  that  linkage  hinges  on  the  causes  of  failure  to 
perform  the  task  correctly.  Each  essential  behavior  within  the  task  is 
analyzed,  with  the  help  of  a  job  expert  where  necessary,  for  potential 
causes  of  failure:  Is  it  because  the  soldier  doesn't  know  when  to  perform 
a  step?  Is  it  because  the  soldier  doesn't  know  where  to  perform  it?  Is  it 
because  the  soldier  doesn't  know  what  the  end  result  looks  like?  Is  it 
because  the  soldier  doesn't  know  how  to  execute  the  behavior? 

For  each  likely  cause  of  error,  the  correct  location,  or  sequence,  or 
product,  or  procedure  is  identified  and  then  described  in  words  or  pic¬ 
tures.  A  question  is  then  framed,  and,  finally,  real-world  response 
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alternatives  (distractors)  are  selected  to  complete  the  test  Item.  The 
Important  point  is  that  by  considering  these  four  questions  about  each 
aspect  of  task  performance,  we  can  pinpoint  both  what  is  important  to  ask 
in  a  knowledge  test  of  task  performance,  and  how  to  ask  it.  This  procedure 
normally  produces  from  five  to  15  questions,  depending  on  task  complexity, 
that  tap  the  critical  aspects  of  task  performance.  It  also  helps  prevent 
test  questions  that  so  often  are  used  merely  because  they  are  easy  to  ask. 
Knowledge  tests  for  two  Artilleryman  (MOS  13B)  tasks  are  shown  at  Attach¬ 
ment  4. 

Erolrlcal  Evaluation 

The  task  analysis  and  test  development  process  are  designed  to  provide 
content  validity,  via  the  consensus  of  subject  matter  experts,  for  a)  the 
completeness  and  relevance  of  the  task  domain,  b)  the  representativeness 
and  relevance  of  the  task  sample,  and  c)  the  description  of  the  steps  and 
procedures  that  are  entailed  in  the  execution  of  each  task. 

There  are  also  several  steps  in  the  empirical  evaluation  of  the  reli¬ 
ability  and  validity  of  the  task  performance  measures. 

1)  Several  small  pilot  samples  are  being  used  to  obtain  the  initial 
reaction  of  test  takers  and  test  scorers,  to  detect  extreme  response  fre¬ 
quencies,  and  to  estimate  inter-scorer  agreement. 

2)  A  major  field  test  using  150  subjects  per  MOS  will  be  used  to 
determine  the  test-retest  reliability  of  the  hands-on  performance  measure 
and  to  estimate  the  pattern  of  relationship  of  the  hands-on  measures  with 
the  parallel  paper-and-pencil  measures  and  other  criterion  measures  in 
total  Project  A  criterion  arrays. 
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INCORPORATING  NEW  PREDICTORS  INTO  THE 
ACCELERATED  CAT-ASVAB  PROJECT  (ACAP)  SYSTEM 

INTRODUCTION 

IncrLnt.AaUaity  “  d^f ««“  na  will 

««“  risvdftu.fe:t^.  and  ..  used  ao  create 
Service  area  composites* 

:r:“zer1"  on'dcT iMplewentation  will  be 

identified. 

BACKGROUND 

The  Department  of  Detente  (DoD)  b:ing 

administer  ‘k'  “I-f’'*?’  «s  rt.JiSn.ible  for  developing 

Se  J^^irirre^onSe  tbe%ventu.l  procurement  and  fielding  of  a  OAT 
system  for  full-scale  testing. 

■As  originally  conceived,  the  CAT  Project  consisted 
designed  testing  ‘“required  tSt^Ly  CAT  hardware  have 

ffiracroL^s^ruarbeW^^ 

in  late  1984  the  "Kfi?^er'tbr“i  J^^'t'a 

A^elerated  «T.m«^ro3^_ 

4thei^  .4c4lly  designed  ““ing  c^puMraJor  ^eter^ 

SoS:Uon1o;  thr^v4t^;“ca!n:veirp:ent  IfSD,  of  CAT-ASVAB  for 
nationwide  implementation. 

in  November  1985.  the  Future  Seating  Co.rtt«e  "'“f 

“r:“c“ow'’Sg“eXed  b5%rservlcea .  This  coa.dtt«  was  charged 

"‘'''■”ee“‘pu?p”S  ^rto^^Wec^^en^tlOTf^  pjSing  selection  and 
'ctn^ifnaurs:::.  4dno"identif,  and  address  issues  related  to 
implementing  new  predictors  with  ACAP. 


NEW  PREDICTORS 

t..t.  «e  important  *0. 

Shlle  OAI-ASVAB  testing  cm  et  el..  1985).  little 

reliable  as  conventional  PP  the  mode  of  administration.  At  the 

benefit  can  be  derived  that  CAT  reliabilities  may  show  only 

composite  level*  Divgi  (  )  nencil  (PP-ASVAB)  composite 

slight  improvements  because  pape  -  -P  ^  measurement  precision  from 

^e  beet  prospect  for  intten.«.l  «^tp  m“e”°gll. 

in  validity  for  test  tw  mbstkntial  Increases  In 

raurty  irTn^siS-crm^lnl;  such  as  tenperanent.  psychonotor. 
and  vocational  interests. 

interest  In  expanding  the  ^f-^-^'.remldnrrtsmrcJ'SfoS  S 
the  last  few  years.  This  .®ograme  are  the  Air  Force  Project 

S^!'t;e”li^“iore«'A!  md  new  predictor  development  efforts  by  the 
Navy.^ 

Project  LAMP 

The  U.S.  Air  Force  Learning  cUssificatiorin^he  Air  Force. 

LAMP)  is  designed  to  parameter s^ of  learning  ability,  develop 

Its  goals  are  to  ^®  Ability,  and  investigate  the  feasibility  of 

rp"a  cog""::  ^odel-based  system  to  psychological  assessment 
(Kyllonen,  1985). 

TnformatinTi  Processing  Speed.  ^^®^  approaches  are 

associated  with  general  cognitive  correlates  approach  and 

used  for  studying  processing  ^P®®*^'^^®  berg  1981;  Pellegrino  and  Glaser, 
the  cognitive  components  ^^g®  oach  individuals  attempt  tasks 

1979).  In  the  cognitive  operations  (e.g..  choice 

which  are  hypothesized  o  comnlex  cognitive  processing.  Project  LAMP 

reaction  time)  underlying  ®°J®^°®P^roroceseing  tasks*  choice  reaction 
utilizes  several  types  of  information  p  .jr-ifonen  1985).  These  tasks 
time,  letter  matching.  eklil.  that  are 

represent  a  hetrogeneous  mix  of  P  Ld  Yantis.  1982).  In 

related  to  mental  ability  ^^®g®^“jg^^g^berg,  *1981) ’.  individuals  attempt 

r^rm^rre'UetfdTarS^^^^^  charactlrlatlce  of  the  te.t 

1a  detailed  dlscueslon  of  these  programs  can  be  found  In  Park  (1988). 
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and  observing  the  effects  of  the  presence  ^  tests  involve 

characteristics  on  test  performance.  Many  of  the  Project  te 

the  oaired-associates  paradigm  in  which  the  memory  load,  and  visual  or 
vtrbafcomnlexitierLe  systematically  varied  (Christal.  1985;  Kyllonen. 
1984)!  HeS^ie  object  is  to  isolate'the  particular  information  processing 
components  that  are  fundamental  to  task  performance. 

Project  A 

The  U.S.  Army  is  developing  an  expanded  predictor  battery 
auspices  of  Project  A  (Campbell.  1986).  The  goal  of  Project  A  is  to 
evaluate  and  upLte  the  selection  and  classification  procedures 
Under  Project  A,  a  comprehensive  set  of  job  L 

developed  to  assess  the  validity  of  the  ASVAB  along  ^^i^h  a  set  of 
experimental  predictor  measures.  These  tests  are  a  spatial  t 

bakery,  inventories  measuring  temperament,  biodata. 

interests,  and  a  battery  of  computerized  perceptual /psychomotor  abilities. 

s««tlal  domain.  The  Project  A  paper-and-pencil  spatial  ability  ^Jttery 
covers  three  facets  of  spatial  ability:  spatial  orientation,  or  the  ability 
to  imacine  the  appearance  of  an  object  from  several  perspectives;  spatial 
relations  or  the  ability  to  mentally  move  objects,  such  as  mental  rotation, 
vi“.U«tion.  or  tho  .Mlit,  to  Io»8ino  the  relative  movements 

of  several  objects  in  a  complex  visual  task. 

Computerized  Perceptual /PsYchomotor.  Project  A  incorporated  the 

cognitive  correlates  approach  in  the  computerized  a . ment  of  the 

following  construct  domains:  perceptual  speed  and  1981) 

reasoning.  These  domains  use  the  reaction  time  paradigm  (Sternberg,  198  ), 
^ircSiects  time  measure,  as  the  dependent  varUble  Reaction 
paradigms  which  are  associated  with  the  cognitive  correlates 
Choice  reaction  time,  letter  matching,  memory  search,  ^nd  sentence 
verification • 

The  Project  A  computerized  battery  was  also  developed  to  measure  three 
psychomotor  ability  constructs  (Imhoff  and  Levine. 
steadiness,  multilimb  coordination,  and  movement  judgement.  J^^® 
involved  such  tasks  as  tracking  a  moving  target  or  projecting  a  mis 
toward  a  moving  target. 

Temperament  /Biodata .  Project  A  included  the  ®®a®"’^®“®*^^ 
temperament,  biodata,  and  interests  as  two 

AssLsment  of  Background  and  Life  Experiences  (ABLE)  and  the  Amy  Vocational 
Interest  Career  Examination  (AVOICE).  The  ABLE  f 

criterion-revelant  attributes  of  temperament,  and  assessed  life  history  an 
Physical  Condition.  The  AVOICE  is  a  version  of  the  U.S.  Air  Force  « 
Vocational  Interest  Career  Examination  (VOICE)  (Alley  and  *  ,  * 

modified  to  be  content  valid  for  Amy  occupations.  It  measured  g®»®J®J; 
interests,  expressed  interests,  and  compatibility  with  the  organizational 

environment . 
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WavY  Program 

Within  the  context  of  new  test  development,  the  U.S.  Navy 
,-4o4-i7  of  nredictors  for  job  performance  measurement.  These  tests  are 
^oluy  cL^uterlxid  sp:tU^  perJept^^  and  cognitive  processing  ^sts  and 
^nplude  a  oaner-and-pencil  biodata  inventory.  The  Navy  work  program  is 
Greeted  Into  four  dl.tlnct  effort,  i  the  Or.phio 

Tests  (GRIP),  Spatial  Abilities,  Cognitive  Speed,  and  the  Armed  Service 
Applicant  Profile  (ASAP)  inventory. 

GRIP.  The  computerized  GRIP  battery  was  originally 
analog  to  work  samples  (Cory,  1977).  The  focus  was  on  simulating  tasks 
in  job  performance  for  the  purpose  of  developing  training  simulation 

systems  • 

<?naHaI  Tests.  Under  the  auspices  of  the  Computerized  Testing 
I.chnoKgf^rn^»  .wtlc  .nd  oniLfd  te.t.  of  .patial-vi.ual  rea.on^g 
beef  deveiopeJ  to  fully  capitalize  on  the  oonputer-a  capability  for 
^  test  administration  (Wolfe  et  al.#  1987)* 

rnanitive  Soeed.  The  Cognitive  Speed  project  is  investigating _ ^ 

information  processing  speed  as  a  distinct  dimension  correlated  to  general 
intelligence  (Saccuzzo,  Larson,  and  Rimland,  1986). 

asAP.  The  Armed  Services  Applicant  Profile  (AS^)  j 

instr^^t  containing  a  heterogenous  set  of  ^  „ 

was  desiened  as  a  screen  for  selection  into  military  enlisted  sej^i^e, 
substitute  for  the  selection  standard  of  a  high  school  diploma  (Trent, 

1987). 

Other  Service  Activities 

Predictor  development  remains  a  Service-level  activity. 
services*  responsibility  to  provide  the  data  on  any 
Committee  to  evaluate.  The  criteria  that  the  committee  “PPji®®  ^ 
evaluating  the  candidate  test  are  shown  in  the 

Anoendix  A.  and  in  Attachment  A  as  the  -ACAP  Future  Tests  Check  List  .  In 
tlTuioVthe  software  for  the  test  should  be  available  to  the  committee  for 

evaluation* 

The  Services  have  developed  software  for  running  a  number  of  their  o^ 
predictors  on  the  ACAP  equipment.  The  largest  effort  has  been  carried  out 
by  Navy  Personnel  Research  and  Development  Center  personal 

pLdiSors  and  a  stand-alone  driver  have  been  programmed  (J.  Y  y 
?ommunlc.tlon.  January  1988).  The  Job  Performance 
Group  sponsored  an  effort  to  convert  seven  spatial  tests  and 
driver  (R.  Harris,  personal  communication,  April  1988).  ^^na  y. 

Research  Institute  (ARI)  has  converted  two  spatial  tests  to  the  sys 

Validation  efforts  on  new  predictors  are  a  mixed  hag. 
effort  has  been  by  ARI,  a  concurrent  validation  of  the  J® 

battery  against  job  performance  measures  in  19  representative  Military 
^cu^utSl  SpeciaUlss  (MOS)  (N  -  9.300).  Although  AEI  plans  to  adapt 
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Project  A  tests  tet^L^thtirn^'^predic 

carried  out  using  the  system.  NPMC  h  p  validate  a  trimmed 

on  the  ACAP  May  1988).  Both 

predictor  battery  (D.  ^  Laboratory  are  concentrating  on 

HPRDC  and  the  Air  Force  Human  Resourc  ,  HP-IPCs.  Data  have  been 
validating  the  JPM  test  battery  on  s  .  Service  specific  performance 

collected  on  the  .ev»  ^  pro'tforbeinE  eLlpted.  The 

and  training  measures.  Th  j--4.-v4na  Its  own  JPM  study  on  four 
Center  tor  Seval  Analyses  is  j^^try  OA  is  underway  (P. 

occupational  Areas  (OA).  Data  ^  Infantry 

Mayberry,  personal  communication.  May 

EXPANDABILITY  OF  ACAP 

The  hardware  «ul  software  of  f U^dLcusSl 

kinds  of  tests  that  could  supplement  the  system.  Botn  wii 

below. 

Computerited  ACAP  testing  ha. 

CAT-ASVAB  Network  (DCN)  mode  of  Stations.  Second,  the 

to  applicants  stationed  at  several  Ei^nee  lest  (Bi)  a  operate 

«.nd?alone  mode  of  operation  is  required  in  c.se^the  ^N 

properly,  or  “ati^ln  the  Mtw^rking  and  stand-alone  modes  of 

SrtSrwL:  ttoroShtrStscrihed  earlier  hy  dones-J^es  and  Rafacr 
aSSS)  ^nd  are  summarised  in  Park  (1987)  (Attachment  B). 

The  desip  concept  ™  sraHrLJ^e^Snr"^ 

being  loaded  into  the  Test  A^  Stations.  In  the  standalone 

broadcast,  subtest  by  subtest,  Station.  One  hardware  constraint  is 

mode,  the  TA  Station  functions  as  an  ET  Station,  Stations 

the  RAM  limits  of  the  system  ^  Under  the  current  design 

hold  2.5  and  1.5  megabytes  of  RAM,  respectiv^  capacity  of  the  system  (B. 
concept,  the  CAT-ASVAB  uses  nearly  the  capacity 

Rafazc,  personal  communication,  January  1988). 

Another  hardware  constraint  is  the  ACAP  f 

device  precludes  using  ‘Jhe.rpsyc^otor  tests  require  special 

™ia»  ever,  and  J-^teitS:  SSrtfha 

t-  the 

examinee  uses. 

in  addition  to  con^raints  imposed  Dy^ar^are.^^^^^^^ 
software  is  an  issue,  •^y  .  gtiig  flow  to  transition  smoothly 

rw.rM!L-A^”.nd  «r  ” 

U»rco!lec'tl^“rtrd”r  t.«-’interrupt  and  -help-  function  features,  and 
downloading  the  data . 
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Several  design  configurations  for  adding  a  FT  battery  to  the  ACAP  LCN 
have  been  discussed  in  greater  detail  elsewhere  (Park,  1987).  These  designs 
either:  (1)  treat  the  FT  as  another  ASVAB-like  subtest  module  within  th 
testing  systemi  (2)  treat  the  FI  as  other  ASVAB-like 
reouire  the  capability  to  provide  a  second  broadcast  of  the  FTs  and 

overwrite  on  RAM;  and  (3)  treat  the  FTs  as  an  2th  Slv  on^ 

not  interact  with  the  LCN.  The  first  design  may  be  workable  with  only  one 

or  two  tests  using  the  current  testing  system.  It  is  meant  ^ 
minimal  software  revisions,  but  the  RAM  restrictions  of  the  hardware 
severely  limit  the  selection  of  tests.  In  the  second  design,  the  FT 
balte^is  just  as  much  a  part  of  the  ACAP  as  is  the  CAT-ASV^.  f^atbvtf 
would  require  additional  software  support  resources,  and  another  1  megabyte 
of  RAM  on  the  TA  Station.  The  third  design  has  the  FT  battery 
separate  disks  at  the  ET  Stations.  This  minimizes  the  software  integration 
support  required  of  NPRDC.  In  all  cases,  system  testing  and  analyzing  the 
Impact  on  changing  operational  testing  procedures  would  need  to  be  done. 

DISCUSSION 


The  efforts  of  the  U.S.  military  Services  to  augment  the  ASV^  have 
been  reviewed.  Common  among  all  projects  is  the  concera  for  attributes 
beyond  verbal  ability,  quantitative  ability,  and  technical  knowledp.  The 
principal  approach  has  been  to  examine  content  domains  not  currently  in  the 
ASVAB  for  their  potential  to  increase  validity  for  predicting  training  or 
job  performance.  These  non-ASVAB  domains  include  temperament  and  biodata 
content  areas;  and  spatial,  perceptual,  and  psychomotor  abilities. 


Integration  of  the  FT  systems  into  the  ACAP  system  is  a  real  problem. 
This  problem  can  be  broken  into  concerns  about:  (1)  software,  (2)  hardware 
and  (3)  dedicated  time. 


(1)  Software.  Systems  integration  of  FTs  with  the  ACAP  software 
is  best  done  by  the  NPRDC  programming  group  which  controls  the 
ACAP  software.  However,  programming  the  FT  battery  competes  with 
the  other  software  development  demands  on  the  project.  Thus,  the 
Services  which  contribute  to  the  FT  battery  should  consider  easing 
the  burden  by  developing  functional  specifications  from  which  the 
FT  battery  may  be  progrsunmed,  and  by  providing  a  form  of  the  test 
already  coded. 

(2)  Hardware  limits.  If  the  current  ACAP  system  has  insufficient 
RAM  to  include  a  FT  battery,  alternatives  must  be  considered. 

These  include:  increasing  RAM,  rewriting  the  software,  or 
administering  the  FT  battery  separately. 

(3)  Time  limits.  The  maximum  time  limit  for  CAT -ASVAB  with  seeded 
items  should  not  exceed  the  paper-and-pencil  (PP)  ASVAB  testing 
time.  Based  on  data  presented  at  the  April  1988  Psychometric 
Committee  meeting,  the  CAT  time  is  nearly  that  of  the  P?  time. 
Thus,  little  time  is  available  for  administering  FTs.  If  ASVAB 
and  FTs  are  given  in  the  same  testing  session,  either  ASVAB 
subtest  times  must  be  shortened,  or  the  total  testing  session 
lengthened. 
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Appendix  A 


MINUTES 

CAT-ASVAB  FUTURE  TESTING  COMMITTEE 
10  February  19B6 

1  cy>  10  February  1986,  a  CAT-ASVAB  Future  Testing  Conroittee  was  held 

at  the  Navy  Personnel  Research  and  Development  Center,  San  Diego, 

Attend 865  sre  shown  st  Attachment  1*  . 

HEfisSswllSsS' 


3.  The  following  topics  were  discussed. 


a  The  oneoins  programs  and  plans  of  the  four  Services  vas  reviewed . 

The  b-ie^5.-  Sides  8?e  It  Attachment  2.  ARI  is  pursuing  the  transfer  of 

data  collection  purposes. 

b  A  <?eneral  consensus  of  the  eomnittee  roerrbers  was  that  information  on 
attempt  to  reduce  reduridancy. 

c.  The  eomlttee  meters  agreed  to  strive  "^SttiriMtlon. 

on  promising  P‘ ^  f^i  -eadv  to  make  these  recommendations 

Z  «^oforsorsr4^“?fet  rid'aSlvra?  the  reo»»ehdatlons  wlU 
be  discussed  in  future  meetings. 

A  Tt  uas  afT-eed  that  before  plans  could  be  made  as  to  Where 

rMfss  sss Ss- 

ralaJf  JslU"ud'‘?f  X\l  mror^uon  „ade 

available  to  the  committee  in  September  1986. 
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CAT-ASVAB  FUTURE  TESTIH5  TECHKOLOGY  COtKITTEE 
6-9  Kay  19B6 

0  • 

^  On  e-Q  Mav  1986.  a  CAT-ASVAB  Future  Testing  Technology  Conaaittee  oeeting 
i;.  ».  ?.««  m  *.v.l  Al.x.ndri.,  VirgiAi.. 

are  ahovn  at  Attachnent  1* 

2.  T«.  ...  cA.lr.4  by  rr  *.i>4olpB  *™J  ***“””S'“*JJ* 

The  other  Service  representatives  verej  Hr  Lonnie  Valentine,  Air 
Sesonrce.  Uhorftory  (ATHIiDi  Ir  ftvl 

Analyses  (CKA);  end  Dr  Robert  Korrison,  Kavy  Peraonnel  Research  and  Develoj^ 
Bent  Center  (NPRDC)» 

3.  The  following  topics  were  discussed. 

a.  The  general  consensus  of  the  FTT  coMittee  denoted  . 

bsttery  as  a  coxfipleaentary  suppleaent  to  the  ASVAB  to  support  differential 
classification.  Validity  to  training  criteria  will  be  improved  by  Beasurint 
abilities  not  tested  now  by  the  ASVAB.  The  nature  of  the  experimental  bat¬ 
tery  will  include  spatial,  perceptual,  and  psychonotor  tests. 

b  The  coBiDittee  discussed  technical  issues  and  concerns  on  the  impl^ 
Bentauon  of  arexperimental  battery  for  the  Accelerated  CAT-ASVAB  Project 
(ACAP).  Ve  met  to  devise  a  strawman  implementation  plan  for  an  experimen  a 
test  battery  on  the  ACAP  system.  The  strawman  plan  delineates  four  phases. 
Phase  I  emphasises  the  investigation  of  multiple 
services.  Data  that  will  be  examined  are:  correlation  with 
tion  with  training  measures,  correlation  with  job  perfomance,  correlations 
with  evaluation/perfomance  rating,  and  uniqueness.  Pnase  II  commences  with 
icip  Here!  proposed  experimental  batteries  are  rotated  as  available  and  when 
d^Th..  h..J  coU.ct.d.  fh...  in  ««n.ne..  .»»  FSB. 

oji  B  large  set  of  tests  to  be  used  as  a  supplement  to  the 

tion  of  parts  of  the  test  will  continue.  Phase  IV  is  the  final  set  to  be 

ised  to  Lpport  differential  classification  by  each  Service  as  they  choose. 

c.  The  committee  recommends  that  90  minutes  be  dedicated  to  the  experi¬ 
mental  battery. 

d.  Dr  Kayberry,  CKA,  briefed  an  update  on  the  CKA  JPK  contract.  A  deci¬ 
sion  for  the  winning  contractor  bad  just  been  made. 

e.  Dr  Petersen,  PDRI,  briefed  the  conceptual  design  of  the  predictor 

battery  for  API  Project  A.  ARI  expects  a  draft  of  ^  of%he 

develS»ent  of  the  trial  battery  by  the  end  of  J Snd  Jf 
results  of  the  concurrent  validation  of  the  predictor  batte^  by  the  and  of 

•Sep  1986.  These  documents  will  be  distributed  to  the  committee  when  availa¬ 
ble. 
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f.  TM  mo  ce=tr.ct. 

Snr^fSinin..  ««  •f.XtV/fU  m*ctnmct  ««  - 

”  :sLn»”n-*r‘uUrT«  co»itt..  ..P».s.a .  «.«»i 

*or  >11  •oaSMleian  «  <0  ■‘O*  '  -  .. 

*.  Hr  »=w..  w»c,  «; 

h.  By  ttoanioous  vote,  .ccomplis^ent  It 

tive  advisory  capacity  a®!"?  '^ion  of  nev  conputerized  predictor 

“«  •«1“«  “•  ““ 

i.  „  .put  vote  (3-..e. 

r^4!  r/;”r;:rr:)r”::or.U'«rc*wo  .j.iU  »„  ,ropos.i  for 

formal  sponsorship*  • 

ft-r  the  BTT  committee  vill  he  held  24-25  July  15- 
in  J-inrcnio!lx“I-liSel,  foUo.in.  tn.  0P>™0  neetin,. 
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MINUrES 

CAT-ASVAB  iUIURE  TESTING  TEfflNOLDGY  OCMTIEE 
6-7  Au^t  1986 

sa. 

Attendees  are  shown  at  Attachsent  1. 

2.  The  »eetl^  was  chaired  by.Ifc. 

«ltSe  (ABI).  tte  Hohert 

3.  Ihe  following  topics  were  discussed: 

a.  Ihe  c<«.lttee  discuss^t^tol 

ploentation  of  the  Svestlaatlon  of  service  specific 

Project  (ACtf).  Hfa  1  ln«l«s  consensus  Sat  tte 

WSgtl^  ^  Sfiiq- crT^StcTdala  «uld  he  best  acc^ 

piished  at  the  KTds  curing  this  liiase. 

b.  Dr.  worn.  Aim.  P«sent^  tte  ^’s"* 

tests  for  the  experimental  battery.  Dr.  Park,  Atu  pres 
eelection. 

^  £U"S.“-.4sr.«  s,. 

bies  of  projects  to  be  placed  .  onalvses  will  be  acconpiished, 

£TsbrS«f..?s^  ~ 

HP* 

3.  The  ASO  eon^«-  SSnf'Swlr^’^f'iSSrlS^ 

g!  ^^"i^^ioJSc^S  StSieflir-iS^erSi  to  rS^«. 


mtKUTES  of  the  2A-25  HOVEMBER  1986 
CAT-S  testing  committee  MEETING 

1.  The  seeting  of^CAT-AS^^ture^^sUng^^itt^^ 

KoveniSser  1986  at  the  ,  jjst  ©f  attendees  is  shovm  at  At- 

4ic.  precedes  presented  »terl.l.  et  W- 

taclsnent  3* 

Ml  rr  omittee  recosnendetions  Mde  below  were  uneninoos. 

2,  preperstlon  oJ  Test  the  individual  Services  be  re^nsible 

The  n  CDSuttee  recoon^s  ttot  toy^vw^^^  ^  ^  rea^y  inte- 

for  prograitming  ^xr  ^  desirSle  that  the  subtest 

pratid  into  ?h;“lntoam  data  needed  for  a  FT 

follow  a  standardized  1/  4tan  code  ID#  response  latency#  and 

siibtest  ate:  sStest  SW  shall  be  transpor^le  to  tte 

:S'^r'vtS':“sot»  ^e  be  Pr-‘S^.^^Ut- 

irs^o^'irS'ST.c^ce'code  in  .  nodular  prosrnwnins  fomat. 

3  Collection  of  Predictor/Criterla  *5!  Cal-RSVW 

■  Ihe  IT  Co«itt«  «^i»t  e«  the  operatlona^ 

be  given  concurrently.  The  ^  adequate  because  of  the 

s.rr.™ , 

be  preferably  switched  off. 

4.  Other  SW  support  to  FT  approved  ACAP  software  for 

seadfn^  ” 

Contnittee  Service  representatives. 

=^‘S‘Si“t^^re:^Sl«lowins  criteria  tn  used  to  evaluate 
candidate  FT  material:  Reliability)  -  (  nr-sguared) 

SI  rnSirnSl‘»Udrt?(w/nain,  «/ 

c)  Reliability 

SI  ^"eSects  (fro.  retest  data, 

f)  Apparent  testing  strategies 

Si  S»n  concurrent  validity  not  n^ailable 

V,  cospletion  rate, 

S!  WntirSSdeoff  of  3)  M 
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7. 


in)  Subgroup  diff6r©no6S 

n)  Differential  validi^  by  consideration  only) 

;i  Inot  iea  for  FSD, 

Tn  «A3ition  to  the  above  criterion#  filler  teste 
^  sSuW  no?  distract  free  dedicated  tests 
(not  interesting/not  boring) 

S)  Be  easy  to  extend  (variable 

t)  Jtore  than  one  variable  length  filler  test 

€.  Privacy  ^  FT  battery,  the  examinee  should  be  informed 

It  IS  noted  that  for  the  ^  oarre  ^  option  of 

i«  >*»"  ,rivac  »ct 

should  be  given. 

?h?*TT  Comittee  ’'^'=^^5(^'to°ex^tiOT'irthat  eSer  languages 
«?e?SS>  af "®  additional  filehandling  problems  are  created. 

8.  Driver  Program  for  FT _ _  ^  initiate  the  driver 

The  FT  Conmittee  reconmends  that  the  w-ftt'  wu* 

program  for  FT. 

“‘n'nrtS'SafSS'following  features  of  the  driver  program  should 

9?,SiS'administratlv.  instructions,  including  Privacy  tet 

b)  define  subtest  order 

c)  check  systan  time  *vrrr«- 

d)  choose  filler  order  if  time  is  short 

e)  bridge  subtests 

f)  autexnate  test  identification 

sSp1??S  2?SJ?te  it«  «hen  'help'  function  is  depressed 
il  Ulster  Datafile  for  examinee  behind  ASVhB 

10.  Information  requestor  Co^  63  ^  available  for  FT 

The  FT  committee  needs  to  know  ^  mo^^ i  or  .chaining  is 

use.  This  needs  to  know:  the  I/O  conventions,  ease  of  re- 

^ntinrseS  Placatant  of  FT  on  the  backup  disk. 

11.  Dedicated  Examinees  ^  t  one-third  of  the  SE  and 

^tSef^n  it»  seeding. 
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E  ~  »  sq?“j.r^3£s  si" 

K^■-«4WSS■M£yiS  “ 

TA* 

mand  is  made  on  the  TA. 

sSK.'u"^^f3a““s‘rs3l"^  u 

if  «r9in,  is  not  «ca»plishea  .t  the  Th 

Station. 

14.  Downloadi^  ^ta  dedicated  FT  subtests  should  be  saved 

E£"  Sffi  ",Sff44Sli‘S^-« 

63)  at  HPPDC. 

Option  B:  Ihe  data  should  be  match/inerged  at  tte  m  Station.  This  will 
S^ire  systers. integration  support  fron  NPRDC  (Code  63). 

#w,4.</h«  r.  ibe  data  will  be  collected  on  separate  disks.  Minimal  SW  sup- 

ing  of  FT  subtests  to  the  ASVAB  later,  prol>ably  at  CRMP. 

“■  H'li'’»tS’^t  Bultipl*  fonts  of  dedicated  FT  subtests  ate  created 
by  changing  the  order  of  FT  to  be  presented. 
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16.  FT  Conmittee  Jtesponse  to  Service  representatives 

Se  FT  Ccrmittee  f  "  t  S^t  arrSasoLbly  expected 

to  submit  the  below  ^  ^  ACS^P  score  Equating  phase,  toch 

.Mission  to  the  BT  0-ee.ttee 

chairman  NLT  1  Jan  87.. 

a)  Uniqueness 

b)  Incrertental  validity 

c)  Reliability 

d)  Sample  sizes  . 

11  -•«  »* 

g)  Tim  1^^  of  ^ting 

h)  Time  length  of  instructions 

V,  ^lp?UTteste  ,£or  coechehllityV 

k)  Subgroup  differences  w/  minority 

l)  ^diHerentlal  validity  rt.oold  be  available 

17.  Current  Plans  of  Action  •  r^nrvaori-  20  Naw  FT  over  to  the  HP. 

HPRK  (^e  «2)  ,SeSd“NSr30  K  h  progrm  driver 

Acconplisbment  of  TOntract  if  time  allows. 

•  will  also  be  accomplished  “"5®^  convert  3  Anry  FT  over  to  the  HP. 

ARI  (PERl-RS)  has  a  S  ^^30  aS^87^.  ARI  (PERI-RS)  has 

Accanplishm^t  of  aSJ  ^  and  a  driver  program  over  to  the 

•H?/T^!ilSnror^trtLJ^  expected  m-T  3»  *pr  «7. 

ihpoSJ  S  S  £^r*ioSSr  m‘.r:^r£or  IT 

yglidity  data  collection. 
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Minutes  of  the  7  July  19 B7  Meeting 
of  the 

Future  Testing  Committee 

1.  TbP  CAT-ASVAB  Future  testing  committee  met  on  ’ 

Diego,  -Cy^.  A  list  of  attendees  is  shown  at  Aten  i.  in  g 

at  Atch  2* 

«« .m" V  ".r*.7tv':iVp..«  ■ »; 

U"S‘w.  .7a“7m  u*7vaU.M.*  »  «’*'f  • 

larger  contraet  to  convert  Jrojeet  A  an  o  xeetlng  ptogrannlng. 

spring  1988.  M.tlBg)  1*  «P*«ed  for 

5el7;e7;*r*»g“'98“.  “^U^SeUv’er^of  1  «=  U.feJ 

it'!;.  R^C.  *  GS*  contract  .111  be 

CNA  is  willing  to  use  any  fraRL  has  no?  current  plans  for  FT 

plans  for  FT  development  or  testing.  AFHRL  has  nor  c  y 

conversion  to  the  HP. 

3.  The  candidate  FT  list  shown  at  Atch  3  was  ^  ^J^Hrilll 

screen  dialoges  (ISDs)  for  most  of  the  tests  ««® . f  11®®*^®^  Service 
be  reviewed  by  the  chairman  and  returned  to  the  sponsor  service 

•  revision  NLT  7  Aug  1987* 

4  The  FT  designs  from  the  ART  working  paper  '*<*g.n* further 

L7a?l«^7l7.‘V.7gnT««t*.7.  "  V/HlV  "*1 

7  Mon  tli  n  are  also  broadcast  to  the  -ET  Stttlons.  The  CAT- 

admlnistratlon,  the  FT  ®^*°  "‘7^--  test  administration  is 

aSVAS  Is  adffiinls tered »  followed  by  the  FT.  efc«*4«»»  th®  ttt 

completed,  the  CAT-ASVAB  and  FT  are  downloaded  to  the  TA  Stttlon.  The  FT 

are  treated  as  part  of  the  axamioee  database# 

This  design  is  limited  to  the  amount  of  Tte  cotmUtle 

ASVAB  and  on  the  amount  of  time  to  be  the  *ET  stttlons,  but 

discussed  the  use  of  2  MB  RAM  boards  Inserted  into  the  ET  stttio  , 

have  questions  about  the  dollar  cost  and  reliability. 

chairman  HLT  20  Aug  1987. 
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Minutes  of  the  19  January  1988  Meeting 
of  the 

Future  Testing  Oomnittee 

1  Die  CAT-ASVAB  Future  Testing  comittee  met  on  19  January  1988  in  San 
Diego,  CA.  A  list  of  attendees  is  shoMn  at  Atch  1.  Die  agenda  is  shown 

at  Atch  2«  . 

2.  Service  research  and  procurenent  efforts  were  revievred.  ARI  has 
procured  25  HP  Int  PCs.  ARI  anticipates  award  of  a  larger  oontrart  later 
this  year  to  ocnvert  Project  A  and  other  Future  Tests  (PTs^  NPRDC  - 
owns  30  HPs  for  Future  Testing.  About  20  Iffs  lack  RAM  boards. 

A  GSA  contract  was  accarplished  to  integrate  the  NERDC's  future  tests. 

3.  AIHRL,  in  a  cooperative  stui^  between  the  MO  and  ID  divisions,  has 
collected  esroerinental  future  t^t  data  on  142  Air  Force  persoram.  Die 
test  measures  Involved  are  the  7  tests  prograimed  in  "C"  un^  Dic3c 
Harris'  contract  with  Eagle  Technology.  Die  subjects  ^  69  Aii^ 
working  in  the  732X0  speciality  and  73  Airman  working  in  the  423X5  ^ 
speciality.  All  142  cases  had  been  subjects  for  WIPT  as^smmt  in  ro  s 
work  in  perfontenoe  i!©asuransnt  in  tlieir  respective  specialities*  AIHRL 
is  assetibliiig  a  datafile  cn  these  cases  vhich  will  include:  (a) 
identification  data;  (b)  experimental  test  item  responses  and  scxores;  (c) 
ASVM  scxjres;  (d)  job  perfoimancse  measures.  AIHRL  %dll  analyze  these  data 
to  ascertain  (a)  ASVAB  validity  for  job  performance,  (b)  experfmm^  test 
validity  for  job  perfomanoe,  (c)  experimental  test  unique  vaiidr^es  an 
the  context  of  ASVAB.  AIHRL  reports  two  observations  from  their  mta  ^ 
collection  experience:  (a)  transportaticn  of  equipment  to  field  si^  iS 
cuthersone.  AIHRL  carried  equiiment  as  "carry-on"  baggage.  ^lane 
regulations  will  be  permit  this  in  the  future,  so  equipent  will  have  to 
be  shipped  in  advance,  (b)  problems  with  equipment  leliabilily,  and  tame 
is  required  for  setup  and  checkout  at  new  locations. 

3.  Die  candidate  FT  list  shown  at  Atch  2  was  reviewed.  Die  toteractive 
screen  ciialoges  (ISDs)  for  these  tests  were  forvrarded  to  Bemie  Rafacz. 

4.  Die  Seirvices  are  asked  to  provide  suitmarles  of  updated  technical 
infonretion  for  each  candidate  test.  Die  summaries  fxjom  Jan  87  were  passed 
out  as  guidance. 

5.  A  draft  of  a  briefing  on  future  testing  was  revievred.  Many 
suggestions  vere  made  ly  the  oororaittee  and  will  be  ijtplemented  ty  the 
chair. 
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ATTACHMENT  A 


ACAP  FUTURE  TESTS  CHECK  LIST 
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Attachment  A 


ACAP  lUlURE  TESTS  CHECK  LIST,  April  1987 


W(  b,  4’*' 


Test  Administration 

A,  Display  format 

1.  similarity  to  CAT  ASVAB* 

2.  character  size:  tib 

3.  font;  nla  ^  ^  ^  ^  .  ,  9 

4.  display  of  time  remaining  in  subtest;  bottom  •  '  . 

5.  display  of  number  of  items  remaining:  bottom 

6.  number  of  items  per  screen  ;  :  **■ 

7.  split  screen  , 

8.  area  and  location  of  graphics 

9.  moving  graphics 

B.  Speed  of  display 

1  •  maximum  delay  for  item  presentation . 

2.  variability  of  delay  between  items:  low 

3.  variability  of  delay  between  subtests; 

4.  variability  of  delay  between  CAT  ASVAB  and  FT  ^t:tery  ^ 

5.  time  to  proctor  call:  same  as  in  pretest,  rounded  vp  to  nearest  10 

seconds. 


C.  E>caminee  input  device 

1,  keyboard  overlay 

2.  response  confirmation; 

D.  Rest  periods 

-  •  1 .  during  subtests : 

2.  between  subtests; 

E.  ,  Omitted  responses 
'1.  how  to  score 

F.  Order  of  subtests 

1 .  ASVAB  before  blocks/FT 

2.  dedicated  tests  before  filler 

3.  multiple  forms 

G.  Maximum  time  limits 

1,  Dedicated  time;  35  minutes 

2,  Sub  test  time  limit 

3,  Time  per  item 

4,  CAT  or  S  controlled  scream  advancing 

5,  out-of-time  stopping  option 

H.  Resxnptlon  of  interrupted  subtest 

1 ,  return  to  Interr^Jted  screen 

2.  data  used/not  \ised  in  scoring 
5  h  :r 

I.  Procedure  at  end  of  subtest  testing 

1 .  dieck  time  for  more  dedicated  tests 

2.  check  time  for  filler  tests 
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ACAP  future  tests  check  UST  (cont.) 


InipleB®itatlon 

A.  initial  operational  test  ami  evaluation  (lOKE) 

1.  sdbject  requirenents 

2.  test  administration  requirenents 

3.  data  requirenents 

B  Checking  performance  of  CAT  system 
*  1 .  FT  impact  on  systen  performance 
«»  keyboard 
—  TA  station 
—  ET  station 

~  interface  with  System  80  ^ 

—  production  of  final  scoring  epo 

c.  Requlren,ents  fot,T««-iroE  ^tlonU  use 
1  .  Adding  new  subtests  to  battery 

2,  checking  validity  of  FT 

—  corr/  with  job  performance 

—  corr/  with  training 
~  corr/  with  ASVAB 

—  corr/  with  other  FT 

3,  checking  reliability  of  FT 

—  test-retest 

retest  interval 

relSblllty  estimate  for  subgroups  and  £^-range  population 

A.  checking  lncl2lon  of  FT  with  service  cmposltes 

Item  pools 
A,  Nunber  of  itens 

‘i5^2fon‘o?aSSft‘fomat  by  s^ce  tectalcal  reps 
2.  selStion  by  service  sponsor  of  IT 

"  fr!ru  Srf of res^faltematlve  »ith  total  score 

n^OTder^^or  random  within  subtest 
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Attachment  B 


Working  Papw 


KS-tTP-87-04 


accelerated  CAT-ASVAB  TEST 

requirements,  addendum  for  examinee  TESTING  STAiiw. 

battery 


Randolph  K.  Park  .  ,  * 

Selection  and  Classification  Technical  Area 


May  1987 


DEIRDRE  KNAPP 

Selection  and  Classification 
Technical  Area 


approved  BY; 

7  LAVIRENCE  M.  BANSER 
Chief,  Selection  and 

Classification  Technical  Area 


conitrutd  St  oH.etst  po»rt.s«  ARI •»  ^ 

tfMltien.  wnitu  se  dttlifvstsd  bt  aihsi  eM'C**  doeumsetstion. 


U.S.  Army  Research  Institute 

for  the  Behavioral  end  Social  Sciences 

eOOl  EU«hhO¥«r  Avenue,  Aleaendrie  VA  22333 
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:  accelerated  CAT-ASVAB  project  (ACAP)  DEVELDRETT  SYSTBl 

FUNCriWAL  REQUIRE  ETTS 
•  ATOETOUM  PDR  EXAMIt^EE  TESTING  STATION: 

SUPPIEETIAL  TEST  .  BATTERY 

Introduction 

SirnttrUed  tests  to  clvlllsn  ^.pUcsnts  J“ee  at 

frifctS  WU^' mtr1^ce''^cS“^s“’s«^  (t®PS)  end  the  associated 
Ibbile  Examining  Team  (MET)  sites.  iHret  the  local 

slitests  using  blocks  of  ASVAB  or  non-ASVAB  items. 

Examinee  Test  Station  Functional  Requirements 

ET  Station  LCN  Itode 

The  networking  mode  of  operatic  consists  of 
vith  a  Test  Administrator’s  (TA’s)  Station.  The  toctiw  of  ^e 
^  to.  fa)  download  the  testing  software  and  data  to  the  ET  Steti^.  W 

2e*M  »e  Ne^ 

Joot-ap  driver  and  the  Tt»* asSSated  software' 

software  requirements  are  shown  at  Table  i .  i 

functions  are  shown  at  Table  2. 
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(Network  Boot-up  Driver) 

A.  Hardware  startup  and  self-test 

B.  Data  acceptance 

1.  ET  Station  software 

2.  Test  Item  banks 

3.  Supporting  data  files 
(Network  ET  Station  Driver) 

C.  Examinee  orientation 

1,  Examinee  log-on 

2.  CAT-ASVAB  introduction 

3,  Privacy  Act  Stateroeit^ 

4.  Examinee  hardware  familiarization 

D.  Test  adninistration 

1 .  General  Directions 

2.  Test  familiarization  and  practice 

3.  Testing  process 

a.  item  selection  and  display 

b.  response  verification 

c.  scoring  a  valid  response 

d.  estimating  ability 

e.  test  termination  rule 

record  response  data  on  backi^  media 

E.  Administering  a  questi^ire 

F.  Personal  data  receipt  frcm  TA  Station 

G.  Personal  data  verification  _ 

H.  Test  data  transmission  to  TA  Station 

I.  Examinee  salutation 


I.  NETWORK  BOOT-UP  progran 

A.  HARTW^RE  START-UP 

A. 

6. 

B.  ACCEPT  ET  DATA 

11,  ET  STATION  DRIVER  program 

A.  ACCEPT  DCAI'HNEE  IDENTimNG  DATA 

B.  EXW'IINEE  ORIEJnATION 

C. 

T) 

C.  ADMINISTER  TEST 

V 

D,  ACCEPT  PERSC^WL  DATA 

r  • 

E.  VERlFf  PERSONAL  DATA 

H. 

F.  ARCHIVE  DATA 
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Network  Boot-Up  Driver  Program 

verTncitlon  on  Hewlett-Pac^a  networking  software  and 

„ea  via  the  AOCEPI  ET  DMA  fonctioo. 

Process.  The 

loapleted.  the  »  S  SIsenT  the  »-a  netwcnrWng 

Software  Is  loaded  dlsoJsion  of  the  stand-alone 

netvrorking  Unk  to  Se  TA  Station  is  active,  the  HP- 

tnode  is  W  below)  f  ®  ^d^^ications  are  established. 

.  IL  networking  software  is  activat^  ^  frcro  the  TA  Station 

The  NEIVJORK  BOOT-UP  program  i®.  pools) .  The  NEIWOHK  BOOT- 

(test  administration  software ,  inf^tabl^.1^  ^  Action.  The  ACCEPT 
ip  driver  proceed,  by  fSr  software 

ET  OiTA  function  Is  »  »“!P^  HP-U.  interface.  If  Ae 

and  supporting  data  fran  the  StatiOT  station  to  the  ET  station 

transmission  of  the  ^  ^  rr  STATIOM  lElVER  and  the  I^EIVJDRK 

■  is  successful,  control  IS  “.^^ssl "  S  mt  successful.  tV  TA 

boot-up  driver  terminates •  If  .rvo  following  softvjare  and 

Soves  the  ET  Stations  fron  t^  We  3. 

supporting  data  now  resides  at  the  ET  btacion  ai.u 


A.  ET  STATIW  DRIVER  program 

B.  HP-IL  Networking  softvare 

C.  Examiner  Instruction  dialogue 

D.  Examinee  dialogue; 

1,  CAT-ASVAB  introduction 

3*.  HardSre^ fsniliarization  and 
i,  Subtest(s)  fsTjiliarization  and  practice 

5.  Salutation 

6.  Error  messages 

E.  Test  Identifying  data 

1 .  Form  tiunber 

2.  Version  nvsiher 

3.  Other 

F.  Test  Data 

1.  Info- tables  . 

2.  Operational  Item  Pools 

3.  Dcperimental  Iten  Pools 
A.  Questionnaires 


t 


Network  ET  Station  Driver  Prograa 


test 


se.  The  NETWORK  ET  STATION  DRIVER  program  Is  to  perf orra  CAT-ASVAB 
iHistratlon, 


Date  (b)  Station  ID  Nimber,  (c)  Site  location,  (d) 

of  sfe?  ®d  biodata),  (g)  E>caninee  Response  File  (\5)loaded  only  f  RESTART 
examinees)  • 

“Th?^wgr»i>  either  proceeds  to  ^Hln  NEW  examinee  Introduction 
%  ^  ®  ^  ppQTAT^T  exdnilnG6  8t  thG  beRinning  of  trxG  xirsc 

Sirs 

sSetSiny^^SekS;^  srsSiS^rrS^srme."Se’'^|^ 

SSrH^riT  activat^  to  provide  the  examinee  an  introduction  to  the  CAT- 

^iUa^izatS^  Ir^ilife 

ss  “°s  "'^“§13 

te^Sg'tafo1S:taf  t^S^Sed^fSo  the  ET 

tSS%e^5  a?l2e-  ^ 

ectl^ItS "itte^  test^  ^t“  W 

ihe  AnamsiER  tem 

s^esffuSy^o^lafiV 

VeTp^els^ard'^a^^U  VE^ 

raSw.  WK  function  will  instruct  the  exasinee  to  verify  the,<5a“- 
^^^e^  verified,  tiie  examinee  will  be  given  a  salutation  and 

will  activate  the  AROilVE  DITA  function, 

Sl£S  V“eS;M;nt^iS^d‘',SS 

les^^rtifeTarrap^Sf/ge  ET  Wl» 

the  next  exaninee’s  identifying  data  to  be  downloaded. 
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ET  Station  Stand-alone  Mode 

The  stind-alone  node  of  cper.tlon  eonslsU 
n  Stetlom  as  dependent  t«tl^  lA 

car.  take  place  on  fte  aeme  atatlm,  if  ^ 

data  mast  be  maniaally  uploaded  ®  ^  oaior  drivers ,  the  Stand- 

-  A— 

station  software  requirenents  are  shovn  in  Table  4. 

Table  A.  r,eneric  ET  Station  Stand-alone  Software  Requi^^ 


(Stand-alone  Boot-iap  Driver) 

A.  Hardware  startxip  and  self-test 

B.  Data  acceptance 

1 .  ET  Station  software 

2.  Test  item  banks 

3.  Supporting  data  files 
(Stand-alone  ET  Station  Driver) 

C.  Examinee  orientation 

1 ,  ExsTiinee  log-on 

2.  CAT-ASVAB  introduction 

3.  Privacy  Act  Stataroent 

4,  Examinee  hardware  familiarization 

D.  Test  administration 

1 .  General  Directions 

2.  Test  familiarization  and  practice 

■'“a!‘?leS'l3“tlo..  and  display 

b.  response  verification 

c.  scoring  a  valid  response 

d.  estimating  ability 

e.  test  termination  rule  . 

f.  record  response  data  on  baclo^  media 

E.  Examinee  salutation 


Stand-alone  Boot-Up  Driver  Program 


UiCM  **sfcw**w  - - — * 

Purpose.  We  vU  We^HTO^  OTAKIUP  function, 

*e  BOOT-UP  driver  progr®  Sa  ^  to  require  »ore 

load  software,  Iten  rte*^S>d-^ine  software  will  perform  more 

interactive  dialosue.  (b)  the  stmd-aloM  aoi  ^  ^ 

verifications  for  the  manual  procedures;  (O  the  Th 
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passcode  Into  each  ET  Station  before  the  data  files  can  be  decoded  and 
l^ded  into  BAH* 

Procesl.  The  TA  inserts  the  ET  BOOT-UP  driver  tatp  Ae  Stttlon 
before  It” is  powered  up.  When  the  ET  Station  is  switched  on,  the  HMffiMARE 
STAKT-UP  function  is  autcraat^.  When  the  HAPJ3WARE  STAKT-UP  Action  is  . 
completed,  the  TA  is  queried  for  the  station  type  (TA  or  ICT),  site  ni^er. 
and  the  desired  operating  configuration  (Network  or  Stand-alone) .  _  After 
the  informatiOT  is  entered  and  verified  by  the  TA,  the  ACCEPT  DATA  Action 
is  activated  and  queries  the  TA  for  the  passcode.  The  ACCEPT  DATA  function 
will  load  the  STAND-ALONE  ET  STATION  DRIVER  from  the  ET  Disk.  The  TA  is 
then  instructed  to  renove  the  ET  Disk  and  insert  the  appropriate  TA  Static 
System  Disk  containing  the  iten  banks  and  supporting  data  files.  When  the 
data  are  successfully  transitiitted,  the  STAND-ALONE  ETT  STATION  DRIVER  is 
activated,  and  the  STAID-ALONE  BOOT-UP  DRIVER  terminates.  . 

Stand-alone  ET  Station  Driver  Program 

Purpose.  The  STAID-ALONE  ET  SI^ION  DRI\^  program  is  to  perform  CAT- 
ASVAB  test  administration. 

Input,  (a)  Date,  (b)  Station  ID  Number,  (c)  Site  lotion,  (d)  (Test) 
FbrnWintifylng  Iteta,  (e)  Examinee  Identifyining  teta  (SSN),  if)  IXarainee 
Response  File  (uploaded  only  for  RESTART  examinees). 

Process.  The  ET  STATION  DRIVER  obtains  the  above  input ^data  from  elth^ 
internal  sources  or  queries  the  TA.  The  data  is  then  logged  the  ET 
Disk.  RESTART  examinees  are  identified  by  matching  the  examinee  s^SSN  to 
an  ID  File  on  the  ET  Disk.  The  TA  is  queried  if  the  RESTART  es^toee  is 
ready  to  complete  testing.  Otherwise,  the  RESTART  examinee  can  be  deleted 

from  the  ID  File  or  tested  later.  ^  _ 

Depending  on  whether  Ae  examinee  is  NEW  or  a  RESTART,  the  program 

proceeds  to  give  NEI  examinee  introduction  procedures,  or  to  initialize  a 
fesTART  examinee  at  the  beginning  of  the  first  nonccnplet^  ^-ASV^  test. 
Fbr  NEvI  examinees,  the  TA  verifies  the  examinee's  SSN  and  biodata  \aa  menu 
drivel  pranpts.  ThB  SS^J,  Form  nurdber,  and  NBW),  entries  are 

created  in  the  ID  File.  An  examinee  response  data  file  is  M^r 

successfully  creating  the  exaiuinee  response  filei  the  (2)  EXAMIip-E 
CSUBJTATION  function  is  activated  to  provide  the  examinee  an  IntroductiOT 
to  the  CAT-ASVAB  testing  program,  fj^^acy  Act  inforoa^^ 
hardware  familiarization  sequence.  Aftervsrds,  the  (3)  AIX>IIN1STER  TEST 

function  is  activated.  v.  .j.  .iv^ 

A  RESTART  examinee  is  reassigned  to  an  ET  Station  by  tlw  tm 

RESTART  exaoinee  response  data  file  will  only  be  fomd  on  tro  ET  Disk. 
When  the  examinee  is  reassigned  to  an  ET  Station,  the  exam  wee  testing, 
infotmation  data  Is  i^loaded  from  the  ET  Disk  into  RAM*  The  sof^^are 
testing  parameters  are  initialized  to  begin  testing  at  the  correct  positin 
in  the  test  administration  process.  The  (3)  AEMllIISTER  TEST  function  is 
activated  to  administer  the  CAT-ASVAB  testing  procedure. 

During  the  ADMINISTER  TEST  function,  the  examinee  response  information 

is  duplicated  in  RAi  and  t^>dated  on  the  ET  Disk  at  tiie  end  of  e&ch  CAT- 
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acuat^  test  When  the  ADMINISTER  TEST  .  function  Is  successfully  couple  ted, 
Ae^See  ^1  be  given  a  salutation  and  CAT-ASV^  the  ET 

Hie  ET  SIATIW  DRIVER  will  bac^  the  vr  station  test 

DiskT  ife.ex^uinee  test  sessK^  tDnes  SteTv^lS 

IS  ^  beared.  The  »  is  then  prompted  for  another 

exaraineCa 

ACAP  Supplenjental  Test  Battery  Functional  Requirements 

2?rfSp^utSt|\«s  StiiS . 

S-svl5%s& 

tIU  Wl^eental 

tests  are  on  an  additional  disk. 

Design  1. 

■n.  Tho  o'Tfll  of  Desi«»n  1  is  to  supplement  the  CAT-ASVAB  with  a 

TT^^er^of  ^?Sts  wit^  major  reconfiguration  of  the  network^ 

supplemental  battery  as  added  subtest  modules. 

Process  The  TA  proceeds  to  boot-up  the  testing  system  as 
NEtSbOCOTOP  driver  will  proceed  '^^^roadcSt’S  CAT-^S  date 

with  the  supplemental  battery  as  shown  at  Table  b. 

Table  5 »  Supplemental  Battery  Test  Materi^ 


G.  Examiner  Instruction  Dialogue 

H.  E>t3ninee  dialogue: 

1 ,  Sijpplemental  battery  introduction 

\\  Slates t(s)  familiarization  and  practice 
A.  Salutation 
5,  Error  messages 

I.  Test  Identifying  date  -  vi^ 

1,  Test  number  (Same  as  f.3,  Table  3) 

J.  Dcperimental  Itea  Pools  (same  as  f.3,  Table  3; 
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Test  adninlstration  of  the  function 

!s.s«. .«.»  -» 

administration  _ _ ,  . . 

1 .  Supplemental  battery  Inttoductlon 

2.  Test  familiarization  and  practice 

3.  Testing  process  , 

a.  item  selection  and  display 

b.  response  verification 

c»  scoring  a  valid  response 

d»  test  termlnatlcn  rule 

e  record  response  data  on  backup  TOedia 

A  general  (D.1.%e^n=a  .Ce) 

be  gfven.  foUo«ed  ^  the  mvacy  Unique  (D.2)‘  test 

supplemental  test  e^Twnces  Since  the  supplemental  tests  are 

familiarization  and  practice  testing 

nonadaptive,  (D.3.a)  1^®“  ®®  fn  3  W  verified  and  (D.3.c)  scored.  The 
process,  valid  res^es  ««  ^^’’i^^^^'^'^le'^lUte^ed,  or  «hm  the 

s*t«t“7«r«  test  ttae  «Pl^- ^  ^ 

recorded  on  backup  oedla  at  ^  succVssfully  completed,  *e 

When  the  AOmNISTER  TEST  &nct^n  is  s^cesst^ty^ 

examinee’s  data  will  consist  of  the  CAT-^SVAJ  ana  ^  ACCEPT 

data.  The  ET  1*en  pwt^  UPU3AD  EXAMINEE  EATA 

PERSONAL  DATA,  VERIFY  PERSOi'lAL  DATA.  ARCHIVE  DA3A,  ana  uri^ 

functions . 

Design  2. 

Purpose.  Thep^seof  ms^lgn^is  «  »^|l“"^^ 

a  iSgi^iuriber  of  tests  ,®A  JLrteoad^tlng  all  tests 

software.  The  f^^_^lf^not  be  possible  if  the  RAM  area 

during  the  ACCEPT  ET  ^‘^^lon^l  Moe^V^s 

available  on  the  ET  .  ^-ation  of  tiie  CM?-ASVAB,  the  RAM  area  for 

at  some  point  test  ^^istr  ^  cleared  to  make  room  for 

rut«  «js&of  aIW 

is  glv®  as  an  example  of  this  design  concept. 

Process.  The  TA  process  ^  self- test, 

NEIVWRK  BOCTIUP  driver  CAT^VAB  and  foil  supplemental 

and  the  data  acceptance  ^  D^lng  the  ACCEPT  ET  DATA 

test  battery  are  loaded  into  ^e  ^  ^tion .  ^  DELETE  TEST 

function,  the  ET^  ^tation^^^grt  from  the  TA  Station  to. 

S^touSSln^th^-ASVM  TM^ftnctlon  is 

S^St'-  A  S^J^i'ssr  U  L^to  ^  TA  Static  to  «tlfy  the  XA 


CAT-ASVAB  testing  is  completed,  and  the  NETWORK  BOOT-UP  program  is 
transmitted  to  the  ET  Station.  The  NEIVTORK  BOOT-UP  driver  proceeds  to  skip 
the  HARUJARE  STARTUP  function  on  condition  that  CAT-ASVAB  testing  is 
completed;  The  ACCEPT  ET  DATA  function  is  activated.  If  CAT-ASVAB  is 
completed  and  supplemental  testing  is  in  progress  (on  the  one  sv^plemental 
test  broadcast  vlth  ]die  CAT-ASVAB),  the  ACCEPT  ET  DATA  function  is  to 
accept  the  remaining  supplemental  battery  test  adoinlstation  software  and 
supporting  data  from  the  TA  Station  via  the  HP-IL  interface.  If 
transmission  of  the  supporting  data  from  the  XA  Station  to  the  ET  Station 
is  successful,  tiie  NETWORK  BOOT-UP  driver  terminates.  If  transmission  is 
interrupted  by  disrupted  testing  (e.g.,  the  examinee  requires  a  RESTART), 
the  transmission  of  ^e  supplenental  battery  is  attempted  again. 

After  the  one  supplenental  test  is  .  completed ,  the  ADMINISTER  TEST 
fimction  continues  with  the  renalnlng  supplemental  tests.  As  with  the  CAT- 
ASVAB  sub  tests,  the  ET  STATION  DRIVER  d\:plicates  the  exaninee  response  data 
in  RAM,  and  x^ates  the  ET  Disk  at  the  end  of  each  test.  _ 

When  the  session  testing  time  limit  is  reached,  the  ADMINISTER  TEST 
function  terminates  testing  on  tiie  current  test.  The  ET  STATlOtJ  DRIVER 
proceeds  as  before  with  the  ACCEPT  PERSaiAL  DATA,  VERIFY  PERSONAL  DATA, 
ARCHIVE  DATA,  and  UPLOAD  EXAMINEE  DATA  functions. 

The  DELETE  TEST  sihfunction  is  activated ,  and  the  RAM  area  holdii^  the 
remaining  portion  of  the  supplemental  battery  is  cleared.  A  transmission 
is  sent  to  the  TA  Station  to  notify  the  TA  that  testing  is  completed,  and 
the  NETWORIv  BOOT-UP  program  is  transmitted  to  the  ET  Station.  The  NETWORK 
BOOT-UP  driver  proceeds  to  skip  the  HARDWARE  STARTUP  function  on  condition 
that  CAT-ASVAB  testing  is  completed.  The  ACCEPT  ET  DATA  function  is 
activated.  If  the  testing  sessicxi  is  completed,  the  ACCEPT  ET  DATA 
function  is  to  accept  the  Mechnical  Ccroprehension  subtest  administration 
software  and  supporting  data  from  the  TA  Station  via  the  HPrIL  Interface. 
If  transmission  from  tiie  TA  Station  to  tiie  ET  Station  is  successful,  the 
NETWORK  BOOT-UP  driver  terminates.  The  ACCEPT  EXAMINEE  IDENTIFYING  DATA 
function  is  called  and  the  ET  Station  waits  for. the  next  examinee's 
identifying  data  to  be  downloaded. 

Design  3. 

Purpose.  The  goal  of  this  design  is  to  supplement  the  CAT-ASVAB  with  a 
large  mrnber  of  tests  resident  on  an  independent  disk  in  the  LCN  node. 
Here  the  design  concept  is  to  build  an  independent  supplemental  battery 
that  does  not  require  interaction  with  the  network  and  thereby  has  nininnin 
software  support  requirouents  from  the  ACAP  software  progranroing  resources. 
This  design  is.  also  applicable  for  supplementing  the  CAT-ASVAB  in  the 
stand-alone  node. 

Process.  The  TA  proceeds  to  boot-up  the  testing  system  as  before.  The 
NETWORK  BOOTUP  DRIVER  will  proceed  with  hardware  start-\jp  and  self-test,  . 
and  the  data  acceptance  function.  Only  the  CAT-ASVAB  software  and  support 
data  are  broadcast  from  the  TA  Station  to  the  ET  Stations.  Test 
administration  proceeds  as  before. 

When  the  UPLOAD  EXAMU^  DATA  function  is  activated,  the  TA  is  notified 
that  the  examinee  personal  data  and  test  administration  response  data  files 

are  being  uploaded  to  the  Tk  Ration  from  the  ET  Disk.  When.  \q>loading  is 
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completed,  the  Tk  manually  replaces  the  ET  Disk  with  the  Supplemental 

Battery.  (SB)  Disk.  . 

Presxmably,  the  SB  Disk  clears  the  CftT-ASVAB  frcxn  the  ET  Station  RAM 
area,  (ffote  that  In  this  acenerio,  edjsequent  CAT-ASVAB  testing  Is  not 
possible  ^ter  the  first  CAT-ASVAB  session.)  The  Input  Data  (%hlch 
incl\ides  Examinee  Identifying  Data  Inpjut  vhen  the  I^EIWORIC  ET  STATICW  DRIVER 
%jas  initiated),  is  written  onto  the  SB  Disk.  The  SB  Disk  dowiloads  the 
supplemental  battery  onto  the  ET  Station  RAM  area.  The  downloaded  software 
and  support  data  are  verified.  The  supplemental  battery  will  be 
adninistered  as  earlier  described.  The  response  data  will  be  record^  on 
the  SB  Disk.  When  the  test  session  time  limit  is  reached,  testing  %nll 
terminate  on  the  current  test. 

Design  4. 

Purpose.  Supplement  the  CAT-ASVAB  with  a  large  number  of  tests  resident 
on  a  separate  disk  in  the  stand-alone  mode. 

Process.  The  TA  proceeds  to  boot-\^  the  ET  Station  in  the  stand-alone 
mode,  and  administer  the  CAT-ASVAB  as  previously  described.  When  the 
examinee  completes  CAT-ASVAB  testing,  the  TA  backs  up  the  exaninee  data  on 
the  TA  disk  and  the  RAli  and  ET  Disks  are  cleared.  The  TA  is  then  prompted 
for  another  examinee  or  for  the  supplemental  battery.  If  the  TA  chooses  to 
test  another  examinee,  CAT-ASVAB  testing  proceeds  as  before.  If  the  TA 
chooses  to  give  the  current  examinee  the  supplemental  battery,  the  TA 
manually  replaces  the  current  disk  (TA  or  ET)  with  the  SB  Disk.  Tie  SB 
Disk  downloads  one  test  at  a  time  onto  the  ET  Station  RAl*!  area.  The 
downloaded  software  and  support  data  are  verified.  The  supplemental  test 
will  be  administered  as  described  earlier.  The  response  data  will  be 
recorded  on  to  the  SB  Disk.  If  the  test  is  completed,  tiie  test  RAM  area  is 
cleared  and  the  next  test  is  downloaded.  If  the  test  session  time  limit  is 
reached,  testing  will  terminate  on  the  current  test.  The  TA  backs  vp  the 
SB  data  on  the  TA  disk,  after  \hic>i  the  RAM  and  SB  Disks  are  cleared.  The 
TA  replaces  the  SB  Disk  with  the  ET  Disk.  The  TA  is  then  prcropted  for 
another  examinee. 


Discussion 

In  order  for  ACAP  to  meet  Its  mission,  several  of  the  Joint-Service 
approved  criteria  for  evaluating  a  CAT-ASVAB  testing  system  were  dropped. 
Among  these  was  the  requirement  for  the  "expansion- flexibility"  of^  the 
testing  system  to  accommodate  all  tests  being  develop>ed  by  the  Services. 
By  selecting  off-the-shelf  test  adninistration  equipment,  the  ACAP  has 
defined  fixed  hardware  limits  on  the  kinds  of  tests  that  can  be 
accommodated.  Ingenious  software  will  be  able  to  expand  these  liinits,  but 
will  be  only  a  partial  response  to  this  concern. 

Within  the  timeframe  of  the  ACAP,  extensive  modification  of  the  CAT- 
ASVAB  software  is  not  possible  without  seriously  disrupting  the  ACAP 
inrolenentation  timelines.  Given  the  level  of  personnel  resources  for 
so*ftware  development  (Rafazc,  1987),  the  ACAP  can  make  only  a  limited 
invesGuent  on  the  supplenental  test  battery. 

These  constraints  point  out  the  Immediate  direction  of  the  ACAP 


^Sppleoental  testa  sev^ely  driving  the 

drav^ack  here  is  t^at  ^  strong  a^antage  to 

tests  because  of  S’  lOI&E  Sch  wiU  t^in  Dec^^  1988. 

it  wav  be  accanplishable  by  AGAT  lu  effort  needed  for  Design  2. 

mS,  DeslgTt  oould  soK  of 

Tjesim  2  illustrates  a  fully  wt  CAT-ASVAB. 

,upp?«Stal  battery  is  Just  ^ 

Ihe  design  has  the  flexibility  —j^o^rational  concerns  (e.g.,  fixed  tes 
i^«ts  to  satisfy  both  research  and  operati^  awiniTtial  amount 

session  time  limits).  ^Hro®®^^'^^LraplSh.  In  addition,  in  to 

of  software  developmeirt  caT-aSVAB  simultaneously  resident  on 

Kave  thfi  suppl€rnGntal  battcty  sn  pat/  sraa  nav  be  rcciuirede 

TA  Station,  another  1  ®  t^ere  the  supplemental  battery  is 

D^ien  3  represents  the  case  ^ere  tne  ^  aupplemental 

configur^ed  at  the  other  ^®^»^.^f^e^estingtysten.  Here  the  major 
battery  is  completely  ^  software  progranning  resources  are 

advantage  is  that  little,  if  any.  t 

gSr^t?U"SaSLuy- handled  ^y  «be  loglstrc 

difficulties.  ,  neslens  3  and  4  present  sceneries. 

Within  the  stand-alone  to  meet  ACAP  lOI^E* 

Design  3  could  represent  a  start  t^^raswy  A 

there  is  a  real  risk  of  rSide  on  different  media. 

as  different  pieces  of  the  ^ta  res  battery  is  by  necessity  on  a 

illustrates  “®f_'t^lrtted*^to  the  testing  protocol.  The  response 

static..  - 

S\mnary 

The  ACAP  testing  ay®'®'  H,Jf’“*'’^l^doajSr*rwlwed  the  functional 
iSVAB  and  on  supplemental  tests,  mis  ^  C«-ASVAB  testing.  In 

i^Stments  for  the  ACM>  H  8“'^  If  a^i^iUntal  test  batt^  «« 

aSltlon.  fo«  ffigSd  ^  tte  ettmt  to  vihlch  the 

described.  The  des^s  diKe^  »  t 

JS5  dlsadv^iteges  of  each  design  «re 

briefly  discussed. 
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The  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  is  a 
multiple  cognitive  abilities  test  battery  used  by  all  the  military 
services  for  selection  and  classification  of  enlisted  personnel. 
Subtest  scores  are  combined  in  several  different  ways  by  each 
military  service.  One  combination,  the  Armed  Forces  Qualification 
Test  (AFQT)  can  be  linked  back  to  assignment  of  military  personnel 
during  the. latter  days  of  World  War  II.  Currently,  all  services 
use  the  AFQT  as  a  primary  selection  hurdle  for  initial  entry. 

Other  combinations  unique  to  each  service  were  developed  to 
predict  success  in  training  in  the  various  military  schools.  The 
Army's  combinations  are  termed  Aptitude  Area  (AA)  composites. 
Individuals  wishing  to  join  the  Army  must  achieve  minimal  scores 
in  one  or  more  AA  to  be  eligible  to  begin  training  in  a  specific 
Army  school. 

Within  the  past  decade,  however,  the  performance  criterion 
favored  by  Army  personnel  managers  has  shifted  from  training 
performance  to  hands-on  or  job  performance.  This  shift  is  a 
reflection  of  three  historical  trends; 

A  shift  in  training  assessment  from  normative  reference  to 
criterion  reference  (i.e..  Instructional  System  Development 
or  ISD); 

The  misnorming  of  one  set  of  forms  of  the  ASVAB  during 

1 976-1 9B0; 

The  introduction  of  ASVAB  testing  into  high  schools  for  the 
purpose  of  recruiting  with  the  "price"  to  the  services  of 
providing  vocational  counseling  information  to  the 
students. 

An  additional  issue  is  the  attention  being  paid  to  all  cognitive 
ability  tests  concerning  their  fairness  to  minority  groups  and  to 
women. 
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Research  Problem 


ASVAB  Forms  8/9/10  were  introduced  in  October  1980;  current 
plans  are  to  have  new  forms  replace  them  in  October  1983.  The 
ASVAB  high  school  testing  program  is  presently  using  (a  correctly 
normed)  ASVAB  5$  but  one  of  the  current  ASVAB  forms  will  be 
transferred  for  high  school  use.  The  Army  uses  both  training  and 
job  performance  (through  its  Skill  Qualification  Test  -  SQT) 
for  test  validation.  However,  to  provide  timely  information,  it  was 
judged  necessary  for  all  military  services  to  provide  training 
validity  information  for  the  preparation  of  the  new  high  school 
Counselor’s  Manual  and  the  technical  manual  for  ASVAB  8-1^.  The 
research  described  below  was  completed  to  meet  that  requirement. 


Method 


Predictors 


ASVAB  8/9/10  is  composed  of  ten  subtests:  General  Science 
(GS),  Arithmetic  Reasoning  (AR),  Word  Knowledge  (WK) ,  Paragraph 
Comprehension  (PC),  Numerical  Operations  (NO),  Coding  Speed  (CS), 
Auto/Shop  Information  (AS),  Mathematics  Knowledge  (MK),  Mechanical 
Comprehension  (MC),  and  Electronics  Information  (El).  For 
purposes  of  Army  selection  and  classification  these  subtests  are 
combined  into  aptitude  area  (AA)  composites  (see  Table  1). 


Insert  Table  1  about  here. 


Within  the  Army  Composites,  the  AFQT  is  used  for  the  initial 
selection  of  personnel  and  the  other  composites  are  used  for  the 
assignment  of  soldiers  to  the  various  MOS  (military  occupational 
specialties)  or  jobs  within  the  Army. 

Table  1  also  shows  the  composition  of  the  newly  proposed 
ASVAB  high  school  composites.  These  composites  will  be  used  for 
career  counseling  and  with  those  students  who  are  considering  a 
military  career.  For  the  purposes  of  this  research  all  ASVAB 
subtest  scores  were  converted  to  standard  scores.  These  standard 
scores  were  then  combined,  as  in  Table  1,  to  form  the  appropriate 
composite  scores. 
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Criteria  and  Sample 

a  soldllr*3“pe?sonne!*'file“"Fj?  grades  on 

1981  the  Army  Research  Institute  (ARI)  collected*'f 

all  MOS  with  100  or  more  entrants  data  for 

iata  Here  end-of-oou?se  grades  ‘hese 

terminated  at  1000  for  the  high  den«ir^  uoe*^^****  Collection  was 
«ar  for  the  remaining  mSI.  ft  JflheL  *»« 

ihloh  formed  the  criterion  meaLrL  for 

)ossible  to  find  useful  criteria  for  an  une  fcaearch.  It  was  not 
iufflclent  variance  in  the  end-offro,.r^^  e*>ow 

issessment  of  predictor  validities  For  ?®  useful  in  the 

>2%  of  the  grades  reported  we^o  a? %h!  in  the  MOS  16E, 

nalyses  of  this  resLrch  we^e  theJefo?r 

ad  a  fairly  large  S  (o%Jano\al°lVd%llnld%%*1o'’o%°%“r“ 

tatistios  for  the  criterion  measure!  from  tS«!  JJ^lrell^Jn'^n 


Insert  Tables  2  and  3  about  hereT 


lalyses 

Iiting\efore^any^Jalidation*^aLi”  ^®^i®  3  required  further 

d  academic  recycles  were  assli^nfi  »  minimum  passing  score 

volving  pass/  fail  trainintr  validation  research 

denying  considerations  befwe°!!‘lSL%%"!o"?ni!'!'|r°ou!!f‘'*''*"‘ 

aau!«'‘f”J%«h'’5o!-°*?TOT“®!!e''!“‘’^‘®-  ‘'■®  orlteria 

the 

■St"  fit  of  the  ASVAB  subtest!  can  be  Interpreted  as  the 

d^%»!!i/Ta?iS!tf!!<  hV"-” 

ntical  to  thaf  n».«  be  shown  to  be  mathematically 

nticai  to  that  proposed  by  Gulliksen  (1950).  ^ 
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Finally,  whenever  the  N  within  an  MOS  was  sufficiently  large 
to  perform  meaningful  subgroup  analyses,  the  above  procedures  were 
repeated  for  subgroups  within  the  MOS.  The  variables  of  race  and 
gender  were  used  in  the  definition  of  these  subgroups. 


Results  and  Discussion 


The  results  and  discussion  of  this  research  will  be  divided 
into  three  topics:  AA  composites,  subgroup  analyses,  and  high 
school  composites. 


Operational  Army  Composites. 

Table  4  presents  the  validity  coefficients  obtained  from  each 
of  the  11  MOS  for  both  AFQT  and  the  appropriate  AA  composite.  For 
each  validity  coefficient,  the  uncorrected  and  corrected  value  for 
restriction  in  range  has  been  computed.  Also  reported  for  each 
MOS  is  the  uncorrected  stepwise  best  fit  estimate  based  upon  all 

10  subtests  of  the  ASVAB.  Inspection  of  the  uncorrected  validity 
coefficients  for  the  stepwise  best  fit  analysis  reveals  that  in 
all  cases,  these  values  are  higher  than  for  either  the  AFQT  or  for 
the  corresponding  AA  composite.  The  average  increment  among  the 

11  MOS  for  the  uncorrected  stepwise  values  in  comparison  to  the  AA 
composite  value  was  .10. 


Insert  Table -4  about  here. 


Somewhat  surprising  is  that,  for  four  of  the  11  MOS,  AFQT 
yielded  a  higher  corrected  validity  coefficient  than  did  the 
corresponding  Army  composite.  In  no  instance  was  the  increase 
greater  than  .08,  as  in  76P.  Correspondingly,  the  increased 
predictive  validity  for  the  Army  composites  in  relation  to  AFQT 
was  greatest,  in  68J  and  67Y,  where  the  increase  was  .11  and  .09, 
respectively. 

Inspection  of  the  validity  coefficients  for  the  Array 
composites  corrected  for  restriction  in  range,  revealed  validities 
ranging  from  .36  for  16P  to  .87  for  33S  with  an  average  validity 
coefficient  of  .66.  The  average  corrected  validity  coefficient 
for  AFQT  was  .64.  The  largest  validities  were  obtained  for  the 
Skilled  Technical(ST)  composite  (.87)  and  for  Surveillance/ 
CommunicationsCS/C)  composite  (.79)  and  the  lowest  average  validity 
was  for  the  Operators/Food  (0/F)  Composite  (.52). 
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Subgroup  Analyses 


There  were  sufficient  sample  sizes  in  16S  to  examine  the 
validity  coefficients  separately  for  Blacks  and  Whites.  Since  16S 
is  currently  not  available  to  women  because  it  is  a  combat 
specialty,  gender  comparisons  were  not  possible.  However,  in  76P 
it  was  possible  to  examine  both  race  and  gender  differences. 

Table  5  presents  the  uncorrected  and  corrected  validities 
separately  for  Blacks  and  Whites  in  16S  for  AFQT,  the  Operators/ 
Food  composite,  and  the  new  high  school  Academic  Ability 
composite.  For  the  present  Army  0/F  composite  there  was 
relatively  little  difference  between  the  corrected  validity 
coefficients  for  Blacks  and  Whites  (.53  and  .51,  respectively).  A 
somewhat  larger  difference  between  Blacks  and  Whites  was  observed 
for  the  AFQT  (.47  and  .68,  respectively). 


Insert  Table  5  about  here. 


Table  6  presents  the  uncorrected  and  corrected  validities  in 
76P,  separately  for  Blacks  and  Whites  as  well  as  males  and 
females,  on  AFQT,  the  Clerical  composite,  the  high  school  Office 
and  Supply  and  the  high  school  Academic  Ability  composites.  As  in 
16S,  the  corrected  validities  are  somewhat  higher  for  Whites  than 
for  Blacks  and  especially  so  for  White  females.  However,  due  to 
the  relatively  small  sample  size  for  White  females,  this 
difference  should  be  interpreted  with  caution.  It  was  somewhat 
surprising  that  the  Army  Clerical  composite  had  the  lowest 
validity  in  comparison  to  the  other  three  measures. 


Insert  Table  6  about  here. 


High  School  Composites 

This  validation  research  offered  the  opportunity  to  analyze 
the  validity  of  the  recently  proposed  high  school  composites. 
Table  7  presents  the  corrected  and  uncorrected  validities  for 
those  MOS  having  an  AA  composite  either  identical  to  or  very 
similar  to  the  proposed  high  school  composites.  The  high  school 
Electronics/  Electrical  composite  is  identical  to  the  Army's 
Electronics  composite,  thus  the  results  for  32D  and  68J  are 
Identical  to  the  data  presented  in  Table  4.  The  high  school 
Skilled  Services  composite  is  similar  to  the  Army's  Skilled 
Technical  composite  with  the  exception  that  AR  replaces  MK  in  the 
Army  composite  and  General  Science  is  included  in  the  Army's 
composite  but  is  not  present  in  the  high  school  composite.  The 
corrected  validity  coefficient  for  the  high  school  Skilled 
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Services  composite  was  .85  and  the  corrected  High  School  Academic 
Ability  composite  was  .82.  Inspection  of  Table  1  reveals  that  the 
only  difference  in  the  high  school  composite  for  Skilled  Services 
and  Academic  Ability  is  the  addition  of  the  Mechanical 
Comprehension  test  in  the  Skilled  Services  composite;  thus,  the 
corrected  validity  coefficient  of  .82  for  the  Academic  Ability 
composite  was  expected  to  approximate  the  validity  of  the  Skilled 
Services  composite  for  33S. 

The  Mechanical  Trades  high  school  composite  is  similar  to  the 
Army's  Mechanical  Maintenance  composite  with  the  exception  of 
Arithmetic  Reasoning  replacing  Numerical  Operations  in  Mechanical 
Trades.  The  resulting  corrected  validity  in  Table  7  is  the  same 
in  6 IB  for  both  the  Array  and  the  high  school  composites.  For  67Y, 
the  high  school  composites  was  .72,  compared  to  .75  for  the  Array 
composite  displayed  in  Table  4. 


Insert  Table  7  about  here. 


The  high  school  Office  and  Supply  composite  is  similar  to  the 
Army's  Clerical  composite  except  the  high  school  composite 
contains  Mathematical  Knowledge  instead  of  Numerical  Operations. 
For  71D  and  76P  the  high  school  corrected  validities  were  .67  and 
.69,  respectively  compared  to  the  Army's  Clerical  composite 
validities  of  .64  and  .60  as  reported  in  Table  4.  Overall,  the 
mean  corrected  validity  for  the  high  school  composites  was  .71 
compared  to  a  mean  corrected  validity  of  .64  for  the  Academic 
Ability  high  school  composite. 

The  mean  corrected  validity  of  the  high  school  composites  was 
.71  compared  to  .70  for  the  AA  composites  in  the  seven  MOS  where 
the  composites  were  either  identical  or  quite  similar.  There  were 
no  applicable  high  school  composites  which  were  similar  to  the 
Army  composite  in  the  remaining  four  MOS. 

Given  the  limitations,  small  sample  size  and  the  relatively 
small  number  of  the  Army's  MOS  analyzed,  the  results  of  this 
validation  research  were  generally  favorable  with  respect  to  the 
validity  of  ASVAB  8/9/10.  The  overall  corrected  validity 
coefficient  for  the  Army  composites  was  .66.  In  the  two  MOS  where 
it  was  possible  to  analyze  validities  separately  for  Blacks  and 
Whites,  the  average  corrected  validity  coefficient  for  Blacks  was 
.52  and  .62  for  Whites.  In  76P  where  it  was  possible  to  compare 
the  corrected  validity  coefficients  for  males  and  females,  the 
resulting  values  were  .61  and  .58,  respectively.  While  the 
overall  validity  of  the  six  Array  composites  analyzed  was  .66,  the 
average  validity  for  AFQT  across  all  11  MOS  was  .64.  This  result 
suggests  that  the  Army  composites  examined  in  this  research 
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contribute  relatively  little  in  differential  prediction  of 
training  criteria.  This  is  not  surprising  given  the  limited  focus 
of  this  research  to  training  performance  in  a  relatively  few  MOS. 
Ongoing  research  with  a  larger  number  of  more  heterogeneous  MOS, 
using  both  training  and  job  performance  criteria,  is  anticipated 
to  provide  more  definitive  results.  Finally,  the  validity  of  the 
high  school  composites  in  those  MOS  where  they  were  appropriate 
compared  very  favorably  to  the  validity  of  the  Army  composites. 


Table  1 


The  Composition  of  the  ASVAB  Composites 


Operational  Army  Composites 


Electronics 
Opera tors/ Foods 
Surveillance/ Communications 
Motor  Maintenance 
Clerical 
Skilled  Technical 
Combat 
Field  Artillery 
General  Technical 
General  Maintenance 


AFQT 

VE 

+ 

AR 

+ 

.5N0 

(EL) 

= 

AR 

+ 

El 

+ 

MK 

+ 

GS 

(OF) 

- 

NO 

VE 

+ 

MC 

+ 

AS 

(SC) 

NO 

+ 

CS 

VE 

AS 

(MM) 

s 

NO 

+ 

El 

+ 

MC 

+ 

AS 

(CL) 

s 

NO 

+ 

CS 

+ 

VE 

(ST) 

s: 

VE 

+ 

MK 

MC 

+ 

GS 

(CO) 

X 

AR 

+ 

CS 

+ 

MC 

+ 

AS 

(FA) 

AR 

+ 

CS 

MC 

MK 

(GT) 

= 

VE 

+ 

AR 

(GM) 

MK 

+ 

El 

+ 

GS 

+ 

AS 

Proposed  High  School  Composites 


Mechanical  Trades 
Office  and  Supply 
Electronics/ Electrical 
Skilled  Services 
Academic  Ability 


AR  +  MC  +  AS  +  El 
VE  +  CS  +  MK 
AR  +  El  +  MK  +  GS 
AR  +  VE  +  MC 
AR  +  VE 
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Table  2 


MOS  included  in  the  Research 


MOS  Name 


Army 

Composite 


05G 
16P 
16S 
32D 
33S 
61B 
61 C 
67Y 
68J 
71D 
76P 


Signal/Security  Specialist  SC 
Short  Range  Missile  Crewman  OF 
MAMPADS  Crewman  OF 
Tech  Controller  EL 
Electronic  Warfare  Systems  Repairer  ST 
Watercraft  Operator  MM 
Watercraft  Engineer  OF 
Attack  Helicopter  Repairer  MM 
Attack  Fire  Control  Repairer  EL 
Legal  Clerk  CL 
Material  Control  &  Accounting  Specialist  CL 
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Table  3 

Summary  Statistics  for  Training  Criteria 


Training  Score 

Training 

MOS 

N 

Mean 

Score  S  D 

05G 

91 

84 

7.3 

16P 

101 

83 

14.2 

16S 

514 

79 

8.3 

32D 

120 

81 

14.2 

33S 

103 

82 

9.0 

61B 

92 

80 

7.7 

61C 

150 

83 

6.9 

67Y 

137 

83 

6.3 

68J 

128 

86 

6. 1 

71D 

96 

73 

22.9 

76P 

613 

87 

5.1 
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Table  M 


Corrected  and  Uncorrected  Validities 
for  Operational  Array  Composites 


Uncorrected 
Stepwise 
MOS  Best  Fit 


AFQT 

Uncorrected/ 

Corrected 


Army  Composite 
Uncorrected/ 
Corrected 


05G 

16P 

16S 

32D 

33S 

61B 

61C 

67Y 

68J 

71D 

76P 


61 

.55/. 81 

(SC) 

.48/. 79 

28 

.15/. 30 

(OF) 

.21/. 36 

28 

.17/. 40 

(OF) 

.23/. 44 

46 

.44/. 67 

(EL) 

.43/. 67 

66 

.46/. 84 

(ST) 

.56/. 87 

51 

.49/. 69 

(MM) 

.45/. 65 

58 

(OF) 

.45/. 75 

45 

.29/. 66 

(MM) 

.39/. 75 

53 

.28/. 62 

(EL) 

.44/. 73 

41 

.38/. 65 

(CL) 

.27/. 64 

48 

.40/. 68 

(CL) 

.26/, 60 
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Table  5 


Validities  for  16S 
by  Race 

Uncorrected  /  Corrected 


Blacks 

Whites 

n 

159 

333 

AFQT 

.03/. ^7 

.21/. 68 

0/F  Composite 

.16/. 53 

.28/. 51 

Academic  Ability 

.09/. 46 

.22/. 48 
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Table  6 


Validities  for  76P 
by  Race  and  Gender 
Uncorrected  /  Corrected 


Blacks 


Males 


n 

273 

AFQT 

.28/. 69 

CL  component 

.12/. 57 

Academic  Ability 
Composite 

.28/. 69 

Office  and  Supply 
Composite 

.29/. 67 

Females 

n 

116 

AFQT 

.26/. 62 

CL  Composite 

-.02/. 46 

Academic  Ability 
Composite 

.29/. 65 

Office  and  Supply 
Composite 

.17/. 59 

Whites 

143 

.60/. 73 
.47/. 65 
.60/. 74 

.61/. 74 

38 

.51/. 77 
.41/. 69 
.44/. 74 

.36/. 71 
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Table  7 


Uncorrected  /  Corrected  Validities 
for  the  Proposed  High  School  Composites 


High  School 


MOS 

Composite 

05G 

N/A 

16P 

N/A 

16S 

N/A 

3  2D 

Electronics/ Electrical 

33S 

Skilled  Services 

61B 

Mechanical  Trades 

61C 

N/A 

67Y 

Mechanical  Trades 

68J 

Electronics/ Electrical 

71D 

Office  and  Supply 

76P 

Office  and  Supply 

Composite 

Academic 

Validity 

Ability 

Uncorrected/ 

Uncorrected/ 

Corrected 

Corrected 

N/A 

.59/. 82 

N/A 

.12/. 29 

N/A 

.17/. 40 

.M3/.67 

.43/. 67 

.^19/. 85 

.42/. 82 

.47/. 65 

.51/. 70 

N/A 

.46/. 74 

.32/. 72 

.25/. 64 

.44/. 73 

.28/. 63 

.38/. 67 

.39/. 65 

.42/. 69 

.40/. 63 
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SAMPLE  ASVAB  SUBTEST  MEANS  AND  STANDARD  DEVIATIONS 
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P6SPOSETEOEHI  FORWE  '  ■  ‘ 

fOf  u»*  of  thi.  form.  AR  34015;  Ih.  propon.nt  tytncy  It  TAGO.  - - - - - 

EFeRENCe  on  OFFICE  SYMBOL  SUBJECT 

-  Request* for  Validation  Studies  for  Armed  Services  Vocational 

Aptitude  Battery  (ASVAB)  8/9/10 


DAPE-MPA-CS 


PERI-RS 


CMT  1 


Dr.  Martin/de/274-827I 


Reference  DF,  CMT  2,  Paragraph  5,  PERI-RS,  ARl,  6  May  83,  SAB,  Inclosure  1. 
Th.  .tfCed  «orKln,  Paper  (Inelosure  » 

Jp“uea! 

=rom  COL  Zaldo  on  3  May  1983. 

;..s  ™r.; 

success  measures  with  adequate  psychometric  characteristics. 

FOR  THE  COMMANDER:  ^ 


2  Incl 


30TCE  L.  SHIELDS 
Director,  Manpower  and  Personnel 
Research  Laboratory 


na  9AQR 


PREVIOUS  EDITIONS  WILL  BE  USED 


,  ISPOSITION  FORM 

'iPorW  of  t»*i»  form.  tM  AB  340-1 S;  th«  pfopomnt  oatncy  It  TAGO. 


EFERENCE  OR  OFFICE  SYMBOL 

:  ;/• 

Iape-mpa-cs 


o  PERI-RS. 


i  SUBJECT 


Request  for  Validation  Studies 
Vocational  Aptitude  Battery  (ASVABj  8/9/10 


FROM  dape-mpa-cs 


date  3  MAY  1383  cmtT 

Mr,  Ruberton/ina/50836 


The  attached  request  (Enclosure  1)  from  the  Director  of  Personnel  Plans, 
isAF  is  forwarded  for  comments  and  information  to  assist  in  responding  to 

IG  Peek. 

I,  Request  your  comments  be  furnished  to  this  office  by  COB  9  M«y  1983. 

K  pARnonse  for  MG  Schwarzkopf,  which  will  be  prepared  by  this  office,  will 

LlSdl'^you?  cSLentrconcern?ng  published  and/or  official  validation  reports 
for  ASVAB  8/9/10  as  requested  in  paragraph  4  of  enclosure  1. 

FOR  THE  DEPUTY  CHIEF  OF  STAFF  FOR  PERSONNEL: 


Enclosure 


MLLIAM  T.  ZALDO,  III 


^  Colonel,  GS 
T^:  Chief,  Accession  Division 


PERI-RS 


TO:  DAPE-MPA-CS 


FROM:  PERI-RS 


date:  ® 

Dr.  Wxng/dhl/48275 


CMT  2 


1.  Reference  i.  .  '  ' 

a  DP' "Request  for  Validation  Studies  for  ASVAB  8/9/10"  from  COL  Zaldo  on 
if8  March  1983  and  our  response  of  19  April  1983* 

b  DP  "Aptitude  Area  Composites"  from  COL  Zaldo  on  30  Nov  1982  and  our 
response  of.  14  Jan  1983* 

2.  The  Army  Research  Institute  (ARI)  is  currently 

rf  a  S?Lf?LtdlSon  pJjjeot.  The  milestones  for  these  analyses,  which  are 
to  lead  ir revising  or  Verification  of  the  Army  Aptitude  Area  composites,  are 
indicaLd  in  referlnoe  (b)  and  were  the  basis  for  the  negative  response  to  the 
Request  of  reference  (a).  This  negative 

recent  JSSCWG  meeting  as  noted  in  paragraph  3  of  MG  Peek  s  letter. 

3.  As  stated,  the  criterion  of  interest  to  the  Army  in 

validity  studies  is  that  of  job  perform^ce.  J^is  necessarily  imp.ies  that 

soldiers  have  to  complete  training  and  be  on  the  30b  in  tne  iiexa 

peilod  If  time  beforl  it  is  appropriate  to  clSlrl  ll 

analyses  is  not  expected  until  October-December  1983*  _ _ 


AUG  80 
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PREVIOUS  EDITIONS  WILL  BE  USED 


cetsiasaKt  fustisc  on'ici!  n«J-3ii*iii 


CHIEF.  SELECTION&CLAS'SIFICATION  TECH 


PERI-RS 

SUBJECT:  Request  for  Validation  Studies  for  Armed  Services  Vocational  Aptitude  (ASVAB) 

8/9/10 

4.  For  a  number  of  years  Army  training  has  used  an  Instructional  Systems 
Development  (ISD)  approach  which  relies  heavily  on  criterion  referenced 
assessment  of  training  components.  When  a  recruit  completes 
traininq,  s/he  moves  on  to  the  next  component.  The  outcome  of  this  approach 
is  that  school  performance  criteria  are  frequently  binary:  JO/NO  GO.  The 
manor ity  of  recruits  achieve  a  "GO"  eventually,  perhaps  not  in  the  initial 
choice  of  Military  Occupational  Specialty  (MOS) .  'Kius,  training  success 
criteria  can  be  expected  to  be  psychometr ically  weak,  in  that  variance  will  be 
minimal.  Other  possible  criteria  such  as  time  to  completion,  namber  of 
recycles,  transfers,  etc.,  are  also  psychometr ically  inadequate.  For  the  FYSl 
cohort,  ARI  requested  special  information  about  training  success  in  160  MOS, 
Specifically  for  criterion  related  validity  research.  Even  with  these  special 
instructions  for  specified  information,  the  criterion  data  for  roost  of  these 
i60  MOS  were  psychometrically  inadequate. 

5.  ARI  can  provide  partial  inforroation  with  respect  to  the  criterion  related 
validity  of  ASVAB  as  follows:  There  are  currently  two  ARI  technical  reports 
in  the  final  review  and  revision  stages  which  are  related  to  the  validity  of 
ASVAB  8/9/10.  The  first,  a  revising  of  the  Hanser  and  Grafton  . 

presented  to  the  Defense  Advisory  Coromittee  on  Military  Personnel  Testing  in 
thSSS^Ser  of  1982,  is  based  on  soldiers  entering  during  1976-1978  who. were 

administered  ASVAB  6/7.=  Analyses  of  the  data  B^^'parallSl 

of  the  misnorming  as  well  as  predicting  the  v^idity  of  Pooelka 

to  ASVAB  6/7.  The  criteria  used  were  SQTs .  The  second  by  Weltin  and  Popelka, 

addresses  the  predictability  of  training  success  i." 

using  the  CL  composite  from  ASVAB  8/9/10  for  the  FY&l  cohort.  If  the 
revisions  and  reviews  of  these  reports  proceed  as  anticipated,  they  should  be 
Ivlilable  fSr  cUation  by  1  July  1983.  Finally,  there  is  a  modest  number  of 
MOS  in  the  FYSl  cohort  training  base  which  have  training  success 
adequate  psychometric  characteristics.  We  are 

Rossmeissl  and  Martin,  to  describe  what  can  be  determined  from  these  data 
concerning  the  validity  of  ASVAB  8/9/10.  We  expect  this  report  to  be 
c^lUed!  reviewed  and  available  for  citation  by  1  1983.  Nevertheless, 

these  reports,  singly  and  collectively,  do  not  fully  meet  the  Army  s 
reguirement  for  criterion  related  validity  because  only  a  portion  of  the 
criterion  domain  has  been  evaluated.  Only  when  the  referenced  in  (b) 

is  completed  will  the  first  relatively  complete  Army  validity  information  b 

available  on  ASVAB  8/9/10. 

6.-  Although  no  amount  of  forewarning  could  speed  up 

FY81  accession  to  complete  SQT  testing  required  a®  validation  criteria,  more 
advanced  notice  of  such  joint  service  requests  would  be  very  helpful  in 
avoiding  misunderstandings,  research  delays,  or  turbulence  in  ongoing 
programs.  At  the  recent  Tri-Service  Lab  Commanders  and  Technical  Dire 
Conference  at  West  Point  Apr  25-27,  this  topic  was  addressed.  The  group 
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»5ECT:^\eque!f  for  Validation  Studies  for  Armed  Services  Vocational 
Aptitude  Battery  (ASVAB)  8/9/10 

-reea  on  the  ®SioUia/Dirertor^of”Aoeession  Policy 

S^;!MSll?|:Treeano;rahe.  tha  l-i,l?,f“tU’'fh:re  Ss^vtirln^aLe  our 
^  ir^fovl^rifaSy*  »S*;se«^;o?f  to^rcIlER^ana  the  .oint-Service 
roups.  .  . 

OR  THE  COMMANDER; 


2  Iix:l.  V  .  .  :  .' 

L.  DF,  28  Mar. 83 
2.  DP,  14.  *1^  .83 


..JOYCE  L."  SHIELDS  •. 

Dirs-ctor.,.. Manpower  and  Personnel 
~  Re g'earcli' Laboratory  ‘ 


DEPARTMENf'OF  THE  AIR  FORCE' 

MEAOQTJARTERS  UNITED  STATES  AIR  FORCE  •  '  '  . 

WASHINGTON.  D.C.  20330 

.  .  . . 28  APR  1933 


"suwECT-  Request  for.  Validation  Studies  for  Armed  Services  Vocational  ^titude  Battery 
-.(/&AB)  8/9/10  \  -- 

■  ‘to:  Major  General  H.  N.  Schwarzkopf  . 

■  ■  ■■  ■  T.  "  ASVAB  14  is  scheduled  to  replace  the  current  high  school  test  on  1  July 

1984.  Two  Air  Force  contracts,  development  of  a  new.  Counselor's  Manual  and 
use  of  the  Validity  Generalization  'concept  to  relate  AS\'AB  14  to  military  and 
civilian  occupations,  are  critical  to  timely  and  successful  test 
implementation.  A  third  contract,;,  development  of  a  technical  manual  for 
A2yabs  8  through  14,  while  not  critical  for  test  implementation,^  will  provide 
iuportant  si^ort  for  the  technical. aredibility  of  all  AS\^AB  versions. 

2... For  the  past  three  fiscal! years",  the  services  have  been  provided.Joint- 
'"'  ■'Seivice  ASVAB  Research  and  Development  funds  to  conduct  studies  that  validate 
’  the  relationship  between  ASVAB  test  scores  and  training  performance  criteria. 

These  studies  are  required  to. -support  the  three  contracts  mentioned  earlier- 
^  ^d,  perhaps  more  importantly,  to  clearly  demonstrate  the  predictive  utility 

of  our. enlistment  tests..  7’ r  ; -  i—  -  =  ='  -  •  • 

•  ■  •  •  •  . 

■  -  3.  'Attached  is  a  l6  >farch  1983  letter  from  the  Oiaiiman  of  the  Joint-Service 

'  _  Selection  and  Classification  Working  Group  (JSSOVG,  formerly  ASVAB  Working 

-group)  ^  which  requests  the  services  provide  published  technical  validation 
reports  to  the  Air  Force  Human  Resources  Laboratory  (AFHRL)  by  1  May  1985. 
At  the  18-22  April  83  JSSOVG  meeting,  service  technical  representatives 
indicated  they  could  not  meet  this  request  and  in  one  case  would  not  have  the 
required  data  until  Oct  1983.  _  . 

-  4.  To  meet  the  AS\’’AB  14  implementation" schedule  and  provide  a  firm  technial 

foundation  for  all  ASVABs,  we  require  all  published  and/or  official  validation 
reports  for  ASVABs  8/9/10  no  later  than  1  July  1983.  V.’e  would  appreciate  your 
strong  support  of  this  requirement.  It  is  our  understanding  that  validity 
studies  will  be  an  agenda  item  at  our  next  Steering  Cooimittee  meting  -on 
11  ^^ay.  Relevant  reports  should  be  provided  to  ^5ajor  John  helsh,  Chainn^, 
•JSSas’G,  AFMPC/i^CYPT,  Randolph  AFB,  TX  78150.  hJy  action  officer  is  Lt  Col  Jim 
Watson,  MPXOA,  AUTOVON  225-4679. 

KENw'eTH  L.  PEEK,  JR. 

Tflajor  Genera],  USAF 
Director,  Personnel  Plans 


1  Atch 

AFMPC/MPCYPT  Ltr,  16  Mar  83 


REn.y  TO  • 

ATTN  Of;  MPCYPT 


DEPARTMENT  OF  THE  AIR  FORCE 

HEADQUARTERS  AIR  FORCE  MANPOWER  AND  PERSONNEL  CENTER 
RANDOLPH  AIR  FORCE  BASE.  TX  78150 


1 6  I.mR  1583 


Request  for  Validation  Studies  for  ASVAB  8/9/10 


’o^ASYAB  Working  Group  Members  -  Testing  Policy  Representatives 

..1»  .  The  Air  Force  Human  Resources  Laboratory  (AFHRL)  has  recently 
*-  let.  a  contract  with  McFann-Gray  Associates  Inc.  to  develop  a 
.-^technical  manual  that,  will  support  ASVABs.- ,8  through  14.  In  order 
.^ito  complete  .  this  •  effort  in  a  timely,  manner,  :  AFHRL  needs,  to  have 
.-..all  published  technical:,  validation  reports.^  that  address  the 
tc  validities  of  ASVABs  .8/9/10..  .  Request:  this,  material  be  ..provided 
.  •;  to.  Dr.  Malcolm  Ree,  AFHRL/MOA,  Brooks  AFB,  Texas,  78235,  by 
1  May* 83.  .  , 


...i  2*  ;  -  .1  am,.aware  -that  all  ■  the  Services -haye_ /these  necessary 
ICyaiidatipn  studies  in  .various  stages  of  completion.*  .In  the:  event 
,'/fbat  'you  carihot  ,  provide  written  reports  .  .pf  the  studies,  '  please 
-  let 'me  know- when  such  Reports  can  be  made,  available,;  pi.ohg  .with  a 
■"negative  -reply  to'^the*  1  May  ^3  suspense,  .  The  topic  of- ASVABs 
'10-  Validity -■  at'udies  ..  will.  '  be  "an  'Agenda  "item  at  the 

roup  mVeti’hg . 


"Cy'tb:  ~0ASp(M"^&Ll, 
"Dr.Seilman 


;.*JPHN  ;r.'  WELSH,  .  Me^j, 

'“Chairman,  'ASVAB  Working'  Group 


AFHRL/mO  ,  "Lt '  Col  ■'  'Amor 


‘.'ficFannrGray ’.'Associates 
~ Ihc 1 , ‘ Dr .  Shore 


D£P.*.RTr.'2:j  r  or  the  air  FoncE 
m  .\nc:u -.MTS  r»5;  .*.!-«  r:.-5.c£  %••'.?« o-.vjn  Awn  itt^eon-nei  ctt^rzt 
'iin-  A’H  ro?.::E  bass,  tx  teiso 


1  6  MAR  1933 


Co?  KPCYPT 


Reques't  for  Validation  Studies  for  ASVAB  8/9/10 

■’®  ASVAB  V7orking  Group  Members  -  Testing  Policy  Representatives 

,1'.  "  ibe  Air  Force  Human  Resources  Laboratory  (AFKRL)  has  recently 
.let  a  contract  with  McFann-Gray  Associates  Inc.  to  develop  a 
‘technical  manual  that  will  support  ASVABs  8  through  14.  In  order 
to  complete  this  effort  in  a  timely  manner,  AFKRL  needs  to  have 
“all  published  technical  validation . reports  that  address -the 
validities  of  ASVABs  8/9/10. -•  Request  this  material  be. provided 
to'  Dr.  Malcolm  Ree,  AFHRL/MOA,  BrooXs  AFB,  Texas,  78235,  by 
-l.  May  83.  f. 

2.  I  am  aware  that  all  the.  Services  have  these  necessary 
validation  studies  in  various  stages  of  completion.  In  the  event 
that  you  cannot  provide  written  reports  of  the  studies,  please 
let  me  know  when  such  reports  .can  be  made  available,  along  with -a 
negative  reply  to  the;  1  May.  83. suspense.  The  topic  of  ASVABs 
8/9/10  validity  studies ..  will-  . -be  -  an  agenda  item  at  the’ 
VB-19  Apr  93  VJorT^-ng^roup  meeting. 


Cy  to:  OASD{MRA&L),  ' 

Dr.  Sellman 

AFHRL/MO,  Lt  Col  Amor 

McFann-Gray  Associates 
Inc.,  Dr.  Shore 


JOHN  R.  V7ELSH,  Maj,  USAF 
Chairman,  ASVAB  Working  Group 


rV 
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lUJan  83  CMT  2 
Mltch^/drb/l»8275 


PERI-RS  (30  Nov  82) 

SUBJECT:  /Iptltude  Area  Composites  ^ 

TO  Chief ,  Accession  Division  PKM  PEM-RS  ypcRfi®  S?, 

DAP&^A-CS  Mltch^/drt)/ 

1.  Die  Army  Research  Institute  Is*  currently  Involved  In  a  major  research  effort 
direct^  at  the  developnent  and  validation  of  Army  sd.ectlcn  and  classification 
measures.  Die  research  is  designed  to  Improve  existing  and/or  develop  new 
trelnductlon  predictors  of  success  In  Aiw  school  training,  Arny-«i<3e  success, 
an!  perTonnance  of  specific  MDS-rd.ated  duties.  An  evaluation  of  the  aptitude 
area  ccn^sltes  Is  pai^  of  this  effort. 

*2,  Data  necessary  to  evaluate  the  v^ldlty  and  utility  of  the  operational  Army 
aptitude  area  (AA)  con?)osltes  Include  (1)  ASVAB  8/9/10  data  for  Army  applicants 
anl  accessions,  (2)  training  grades  or  training  performance  ratings  for 
accessions,  and  (3)  9QT  and  other  job  performance  measurements  for  recruits. 

While  data  In  (1)  are  curraitly  available  for  many  Fl8l  and  Ff82  ap^lcants, 

*  current  data  in  (2)  are  limited  to  end-of-course  grades  for  seme  PY81  awesslons 
In  sd.ected  MOS.  SQT  data  (3)  are  not  now  available  for  most  PX81  recruits; 
they  will  be  available  for  the  H81  cohort  during  this  fiscal  year*  ^Other 
performance  data  currently  available  are  attrition,  promotion,  and  disciplinary 
data  for  a  limited  nuaber  of  Ff8l  accessions,  but  other  specific  job  perforce 
measures  are  generally  unavailable.  Die  data  described  at  (1) ,  (2) ,  and  C3/ 

'  expected  to  be  essentially  ccn?lete  by  1  October  1983*  Appropriate  analyses  to 
develop^  verify,  and  docunent  an  acceptable  revision  of  aptitude  area  ccx^sltes  . 
will  require  6-9  months,  followed  by  a  3-oQnth  professional  and  le^  re^ew. 

‘  Dius,  It  Is  not  feasible  to  meet  the  1  October  1983  deadline  suggested  above. 

3.  A  more  extensive  set  of  data  win  be  coUected  from  a  sample  of.  the  5^3v^il 
cohort.  Dlls  win  Include  a:  broader  variety  of  ^ 

•  performance  measures,  and  Is  eqpected  to  be  ccrnplled  by  30  September  19o5» 

.TOese  analyses  win  focus  on  (1)  the  rd.l^nity  of  oon^slte  scores  wer 

various  ASVAB  forms  and  administrations,  (2)  the  predictive  validity  of 
.  composite  scores  with  respect  to  performance  In  trali^  (as  measu^  by 
ap^priate  levd.  and  tlme-to-oaster  Indices)  sei^tely  for  hl^  dainty  MOS, 

-  (3)  the  predictive  validity  of  caiposlte  scores  with  respect  to  cn-lhe-job 

-  performance  (as  measured  by  ratings  and  hpds-on 

{^pjh  frequency  a«i  representative  job  tasks)  separately  for  hl^i^^lty  MOS, 

■■  W  the  generallzablllty  of  validity  coefficients  to  tral^  and  job 
performance  for  MOS  not  examined,  (5)  the  differently  vyidlty  - 

SS-tude  area  corposites  with  respect  to  tralnlng/job  performance  for  Individual 

ms.  and  (6)  the  utility,  in  terms  of  Improved  perfor^cy  of  _ _ 

^iJcS^Sasslflcation  decisions  based  on  the  additional  Information  provided 
'  by  an  increasln^y  greater  number  of  AA  cenposites.  Preinduction  preyctor  ^ 
for  performance  In  Army  school  training,  Artny-wl^  success,  5"^?  ^ 

specific  MOS-related  duties  will  be  used  to  verify  and  extend  toe  aptitude  area 
S^site  set-  developed  during  n84  as  described  In  paragraph  2. 


2  Incl. 
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nra-RS  (30N0V  82) 

SUBJEC?!:  iiptltude  Area  Conposltes 

ii.  to  order  to  meet  the  Arny*8  need  for  re-evaluated  AS7AB  aptlt^e  area  scows 
with  appropriate  psychcmetrlc/legal  docunentatlcn,  ARI  proposes  to  exaalne  ASVAB 
8/9/107tralnliK.  and  SQT  data  for  the  J8l  cohort.  A  research  for 
prlunlnaiy  examination  of  the  AA  coiposltes  >iiai  be  b^ed  cn  Fl8l  co^rt  data, 
toe  plan  will  be  developed  by  Ifey  1983*  We  will  coordinate  the  res\£^  of  tMs 
Sadnatlon  with  your  office  as  analyses  are  coipleted  between  ^  Ifey  and  next 
‘Writer#  *  We  expect  the  prd.lmlnary  evaluation  of  the  AA  con^sltes  ^ 
co^ete  by  15^rch  198M.  toe  final  evaluation  shoiild  be  con?ilete  ^  15  Kay 
loSJ.  toe  systen  should  be  available  for  DCSPER  review  1  August  198^* 
-^enentatlS  will  occur  pending  approval  by  IX^ER.  Request  fl®  ^e  te  ^ed 
as  a  milestone  for  changing  the  AA  coDj^s^es.  More  ®*beM^e  analyws,  ^ed 
on  a  much  more  cai?iLete  data  set  for  FX83/81|  recruits,  and  __ 

additional  alteratlcais  to  AA  caiposlte  structure,  should  be  available  for  DCSPER 

review  by  1  August  1986. 


/JOYCE  L.  SHIELDS,  Ph.D. 
Director,  Maipower  and 
J^rsonnd  Research  Laboratory 
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SU  EJECT 

Aptitude  Area  Composites 
FROM  DAPE'MPA 


o^te  so  hoy  - 

Mr.  Ruberton/nJd/50836 


CMTt 


At  present  the  Army  uses  ten  different  Aptitude  Area  Composite  fuhtesV  ^ 

ir^  JI  vocatlonal  Aptitude  Battery  (ASVAB)  for  enlistment  purposes.  The  ASVAB  suhtest 

imposition  of  these  AA  is  linked  to  the  Army 

LCB-73)  i*lch  was  used  prior  to  implementation  of  ASV^  In  1976. 

,  There  are  several  Issues  Involved  with  the  continued  use  of  ten  Army  AA  composites. 

acB-73  end  ASVAB  6/7  (used  IJan  76  -  30  Sep  80)  had  16  subtests  while  ASV^  8/9/10 
...?  .U«  SoT^nTprolectid  foil-™.  fox«l  W  te.  .ubte.t..  Tb.  crr.nt  AA  .v.rl.p 
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PROJECT  A  DATA  BASE  SECURITY: 
AN  UPDATE  AND  A  REMINDER 


PAUL  G.  ROSSMEISSL 
AND 

LAURESS  L.  WISE 


THE  NEED  FOR  SECURITY 

Whenever  a  large  amount  of  data  on  individuals  is  maintained 
and  stored,  it  is  necessary  to  develop  procedures  to  protect  that 
data  from  comprcsnise.  The  security  of  the  Project  A  and  B  L^B 
is  particularly  important  for  a  number  of  reasons.  S^e  or  the 
data  collected  on  individual  soldiers,  such  as  promotions, 
paygrade,  or  disciplinary  actions,  will  be  private  in  nature,  and 
the  privacy  of  that  information  must  be  maintained.  Since 
researchers  will  be  accessing  the  LRDB  for  a  variety  of  uses,  the 
integrity  of  the  data  must  be  maintained  to  ensure  that  the  data 
ranain  accurate  and  consistent  across  uses.  Finally,  it  is 
necessary  to  secure  the  database  to  ensure  that  the  Army 
maintains  ownership  of  the  data.  In  other  words,  to  ensure  that 
the  data  within  the  LRDB  are  used  only  for  authorized  Project  A 
and  B  research. 


SECURITY  PROCEDURES 


The  security  of  the  LRDB  will  be  protected  in  three  ways. 
Soldier  social  security  numbers  (SSNs)  will  te  routinely 
encrypted  to  ensure  the  privacy  of  each  soldier's  records. 

Access  to  the  LRDB  will  be  controlled  both  to  further  protect 
soldier  privacy  and  to  ensure  proper  use  of  the  data.  Finally,  a 
log  will  be  maintained  for  the  LRDB  system  that  will  note  each 
attempted  access  of  the  LRDB  and  whether  the  access  was 
authorized  or  not.  Each  of  these  procedures  will  be  detailed  in 
the  subsections  that  follow. 


SSN  ENCRYPTION 

The  key  aspect  to  guaranteeing  the  privacy  of  individual 
soldier  data  will  be  the  coding  or  encrypting  of  each  soldier's 
identifier.  This  encryption  will  be  accomplished  by  scrambling 
each  soldier's  SSN  in  an  unpredictable  way.  The  algoritta  ttiat 
will  do  the  encrypting  (and  if  needed,  decrypting)  will  be  kno^ 
only  to  the  LRDB  manager  and  ARI  in-house  database  mna^rs.  A 
printed  copy  of  the  algorithm  will  be  securely  maintain^ 

Project  A  data  base  manager.  All  of  the  data  files  of  the  LRDB 
that  can  be  routinely  accessed  and  any  project  workfiles 
generated  from  the  larger  LRDB  will  use  only  this  encrypted  SSN 

as  the  soldier  identifier. 


CONTROLLED  FILE  ACCESS 


The  integrity  and  accuracy  of  the  LRDB  data  will  be 
maintained  by  controlling  the  access  to  the  large  files  or 
relations  within  the  database.  This  procedure  will  also  further 
contribute  to  the  privacy  protection  of  individual  soldier 
records.  In  general,  the  system  to  be  adopted  will  use  the  RACF 
procedures  available  at  NIH  to  allow  the  access  of  particular 
files  to  authorized  users.  Under  RACF,  different  levels  of 
access  can  be  granted  to  different  users.  By  specifying  a 
"universal  access"  of  "NONE"  on  all  of  Project  A  datafiles, 
access  will  be  restricted  to  only  those  users 
exceptions.  In  most  cases,  such  users  will  be  allowed  READ 
access  only.  Such  users  will  have  to  provide  an  eight-^aracter 
RACF  password  (different  for  each  user)  in  order  to  read  the 
datafiles  for  which  they  have  been  given  access.  Using  ^e 
provisions  of  RACF,  a  series  of  access 

Which  should  provide  timely  access  to  relevant  data  m^ed  by 
Project  A  and  B  researchers  and  yet  protect  the  security  an 
integrity  of  the  data. 

Level  1.  At  the  highest  level  of  access  will  be  the 
database  managers.  Currently,  these  individuals  are  Dr.  Lauress 
Wise  and  Ms.  Winnie  Young  of  AIR  and  Dr.  Paul 
Level  1  personnel  will  have  access  to  all  of  the  files  within  t 
database.  Furthermore,  only  I^vel  1  personnel  will  be  able  to 
officially  enter  data  into  the  database  or  modify  data  already 
stored  in  the  database.  Thus,  the  database  managers  must  assume 
responsibility  for  data  entry,  editing,  and  the  storage  of 
original  data  materials  (i.e.,  tapes,  diskettes)  in  a  secure 
location.  In  addition,  it  will  be  the  duty  of  the  level  1 
personnel  to  create  Level  4  workfiles  as  they  are  needed  by  other 
project  researchers. 


Level  2.  Personnel  at  the  second  access  level  will  be  able 
to  directly  read  data  from  all  of  the  files  in  the  database  with 
the  exception  of  the  Link  File  (see  LRDB  Plan  Section  4.3.2.) 
that  contains  basic  soldier  identifying  information.  This 
exception  is  made  to  maintain  soldier  privacy.  Dr.  Lawrence 
Hanser,  the  current  COTR  will  also  have  Level  2  access  as  will 
Ms.  Frances  Grafton  of  ARI. 


Level  3.  Most  project  personnel  will  have  scxne  I^vel  3  LRDB 
access.  Researchers  at  this  level  will  have  direct  access  to  all 
files  that  are  generated  by  the  particular  tasks  they  are 
investigating.  Furthermore,  they  will  have  direct  acTOSs  to  the 
files  created  by  other  tasks  that  directly  impact  their  work. 

For  example.  Task  2  researchers  will  have  direct  access  to  toe 
task  analysis  data  collected  by  Task  5  so  that  toe  i^w  predictors 
that  are  developed  will  address  areas  of  the  criterion  space  not 
currently  covered  by  ASVAB. 


Level  4.  The  most  cointion  vray  in  which  project  researchers 
will  access  the  LRDB  is  through  the  creation  of  workfiles  (see 
LRDB  Plan  Section  4.4.).  By  requesting  the  creation  of  a 
workfile^  a  researcher  will  be  able  to  obtain  data  from  all  of 
the  large  files  in  the  database  except  the  Link  File  which  will 
contain  soldier  identifying  information  and  will  al^ys  ^  kept 
private  and  secure.  The  key  aspect  of  workfiles  relevant  to  LRDB 
security  is  that  the  researcher  will  only  revive  the  data  that 
he  or  she  requested  and  there  will  be  a  precise  record  of  who 
requested  that  data.  When  a  project  scientist  requires  a 
workfile,  he  or  she  will  submit  a  data  request  form  to  either  toe 
contractor  or  ARI  database  manager.  This  request  form  will  ask: 


(1)  Who  wants  the  data? 

(2)  What  variables  are  needed? 

(3)  What  sample  is  needed? 

(4)  Viiich  LRDB  file  or  files  contain  the  data  being 

requested? 

(5)  Vtiy  are  the  data  needed? 

(6)  Will  the  data  be  downloaded  to  hardware  other 

than  the  NIH  computer  facility? 

(7)  What  will  be  done  with  the  data  after  its  current 

use  is  completed  (i.e.,  file  will  be  scratched  or 
saved  for  future  use)? 


Requests  from  Project  A  staff  for  workfiles  can  be  initiated 
by  typing  the  WYLBUR  command  "WORKFILE".  This  command  will 
present  the  user  with  a  workfile  request  form^  that  can  then  be 
WYLBUR  mailed  to  the  data  base  managers  (initials  LWV,  WYO,  or 
GAR).  Such  requests  will  be  reviewed  by  the  LRDB  Manager  and  the 
Task  1  Monitor  for  appropriateness.  If  any  dispute  should  arise, 
the  CX)TR,  Principal  Scientist,  and  relevant  Task  Leader  will 
confer  with  the  LRDB  Manager  and  Task  1  Monitor  to  resolve  the 
dispute. 


Requests  from  researchers  outside  of  Project  A  for  wrkfiles 
will  be  reviewed  by  the  ARI  Task  1  Monitor  and  the  OOTR  for 
approval.  Release  of  data  outside  of  Project  A  ^t  within  ARI  is 
an  ARI  policy  issue.  Requests  for  data  files  originating  outsicte 
of  ARI  must  be  approved  by  ODCSPER  as  required  by  the  Project  A/B 
LOA  with  that  office. 

Whenever  workfiles  are  created,  the  researcher (s)  receiving 
toe  files  will  agree  to  the  following: 


(1)  Ihese  data  are  the  property  of  the  Amy  and  the 
researcher  accepts  responsibility  for  preventing  further  release 
of  these  data  without  fomal  ARI  approval.  This  means,  in 


particular,  that  derivative  files  with  records  on  individual 
soldiers  will  be  protected  by  RACF  or  an  equivalent  system. 

(2)  Papers,  publications,  and  briefing  charts  based  upon 
these  data  will  be  submitted  to  ARI  for  clearance  before  being 
shown  outside  of  Project  A. 

(3)  The  researcher  will  inform  the  LRDB  Manager  when 
analyses  of  the  workfile  have  been  completed  so  that  it  can  be 
purged. 


LFDB  ACCESS  LOG 

A  reoort  will  be  generated  each  week  showing  the  numter  of 
accesses  to  each  LRDB  datafile,  including  each  workfile,  by  each 
authorized  user.  In  addition,  the  monthly  accomting  information 
of  each  project  user  will  be  monitored  for  any  indication  of 
unauthorized  access  to  the  LRDB.  These  audit  trials 
as  a  second  level  of  protection  against  unauthorized  use  of  t 
data  by  anyone  who  manages  to  obtain  the  necessary  ^CF 
passwords.  They  will  not  directly  prevent  unauthorized  access  to 
the  LRDB,  but  the  threat  of  exposure  should  serve  as  a 
significant  deterrent  to  attempts  to  unauthorized  LRDB  access. 

The  log  will  also  help  the  database  managers  decide  which  project 
files  should  be  stored  on  disk  rather  than  tape  by  providing 
information  as  to  how  frequently  data  are  requested  from  any  give 
file. 


OTHER  SECURITY  PROCEDURES 

Much  of  the  data  that  will  be  entered  into  the  LRDB  will 
cOTie  from  existing  Army  sources,  such  as  the  EMF.  Additional 
precautions  beyond  those  mentioned  above  will  be  taken  to  secure 
the  information  on  these  data  tapes.  The  key  aspect  of  this 
additional  security  is  to  collect  and  store  information  from 
these  sources  only  if  it  is  essential  to  the  goals  of  Project  A 
and  B.  For  example,  with  regard  to  the  EMF,  this  LRDB  plan 
indicates  specifically  which  variables  will  be  needed.  O^er 
variables,  in  particular,  each  soldier's  location  and  assigned 
unit,  will  not  be  routinely  acquired  and  stored.  In  addition  to 
limiting  the  data  elements  to  be  stored,  the  ni^r  of  soldiers  _ 
for  vAicxn  any  data  will  be  retained  will  be  limited.  As  indicated 
in  Section  3  above,  the  LRDB  will  not  obtain  and  keep  information 
on  all  active  service  personnel.  Only  data  from  personnel 
selected  for  Project  A  and  B  research  will  be  maintained. 

SUMMARY 

Any  set  of  procedures  designed  to  store  data  electronically 
needs  to  balance  the  ease  with  vdiich  data  can  be  acceded  against 
the  security  of  the  database.  The  procedures  presented  in  tins 
Sction  ter^  to  favor  the  security  aspect  of  this  talanc^  The 
number  of  data  files  that  most  project  researchers  will  be  abl^e 
to  access  directly  will  be  quite  limited.  Furthermore,  only  the 


database  managers  will  be  able  to  add  or  modify  data,  ai^  access 
the  true  soldier  identifying  information.  However,  efficient  use 
and  rapid  creation  of  the  workfiles  should  provide  any  project 
scientist  with  the  data  that  he  or  she  needs  to  perform  the 
required  research. 
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Project  A  Workfile  Request  Form 


Name  - - - - - - - 

NIH  RACF  ID  _ _ _ 

Information  on  the  Data  that  you  need 

1.  From  v^ich  LRDB  file  or  data  collection  do  you  need  data? 

2.  VJhich  variables  from  the  total  data  set  are  needed? 

3.  What  sample  or  subsample  of  the  cases  is  needed? 

4.  V3hy  are  these  data  needed?  (Please  attach  a  short  outline 
of  the  planned  research.) 


5.  Will  the  data  be  downloaded  to  any  system  other  than  the  NIH 
computer  facility? 


6.  What  will  be  done  with  the  data  after  its  current  use  is 
completed  (i.e.,  the  file  will  be  scratched  or  saved  for  future 
use)? 


Please  note  that  as  you  request  a  workfile  you  agree  to  the 
following  policy: 


(1)  Ihese  data  are  the  property  of  the  Army  and  the  re- 
searcher  accepts  responsibility  for  preventing  further  release  of 
these  data  without  formal  ARI  approval.  This  means,  in  ar¬ 
ticular,  that  derivative  files  with  records  on  individual  sol¬ 
diers  will  be  protected  by  RACF  or  an  equivalent  system. 


(2)  Papers,  publications,  and  briefing  charts  based 
these  data  will  be  submitted  to  ARI  for  clearance  before  being 
shown  outside  of  Project  A. 

(3)  Ihe  researcher  will  inform  the  LRDB  Manager  when 
analyses  of  the  workfile  have  been. ccxnple ted  so  that  it  can  be 

purged. 
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This  working  paper  is  an  unofficial  document  intended  for  limited  distribution  to  obtain  comments.  The  views,  opinions,  and/or 
findings  contained  In  this  document  are  those  of  the  author(s)  and  should  not  be  construed  as  the  official  position  of  ARI  or  as  an 
official  Department  of  the  Army  position,  policy,  or  decision,  unless  so  designated  by  other  official  documentation. 


PROJECT  A  PROPONENT  FEEDBACK  BRIEFING 
REVIEWING  STATUS  AND  RESULTS  FOR  MOS  63B: 

JUNE  18,  1987 

Pj^ojsct  A  is  a  coinponsnt  of  a  Iairg6~scal6  of  fort  to 

improve  the  Army's  selection  and  classification  system.  In  1985, 
a  manor  Project  A  data  collection  was  completed  involving  over 
9400  soldiers.  Following  the  completion  of  the  principal  set  of 
analyses  on  these  data,  briefings  were  offered  to  Army  agencies 
who  had  provided  support  to  the  research  and  might  have  some 
interest  in  the  results  obtained.  This  working  paper,  prepared 
to  meet  a  specific  Army  request  for  information,  presents  slides 
and  accompanying  narrative  for  one  such  briefing,  presented  by  an 
Army  Research  Institute  representa:tive,  Michael  Rumsey,  to 
proponents  for  the  Light  Wheel  Vehicle  Mechanic  (63B)  military 
occupational  specialty  and  other  attendees  at  Aberdeen  Proving 
Gj^ound,  Maryland,  on  June  18,  1987.  Where  the  information 
presented  is  no  longer  current,  explanatory  notes  have  been 
inserted. 
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Usefulness  of  Spatial  and  Psychomotor  Tests 
for  Predicting  TOW  and  UCOFT  Gunnery  Performance 

Jay  M.  Silva 


General  Introduction 

This  paper  describes  a  predictor  validation  effort 
undertaken  by  the  Army  Research  Institute  with  the  assistance  of 
the  Centers  and  Schools  of  the  U.  S.  Army  Infantry  at  Fort 
Benning  and  U.  S.  Army  Armor  at  Fort  Knox.  The  goal  was  to 
examine  whether  new  tests,  developed  under  an  on-going  effort  to 
measure  psychomotor  and  spatial  skills,  would  improve  the 
selection  of  trainees  into  gunnery  Military  Occupational 
Specialties  (MOS)  beyond  the  contribution  already  made  possible 
by  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB) . 

The  ASVAB  is  presently  the  major  selection  battery  used  by 
the  Services.  At  the  time  of  enlistment,  Army  trainees  were 
required  to  pass  cognitive  ability  screens  based  on  a  combination 
of  ASVAB  subtests.  The  10  ASVAB  subtests  are; 

Arithmetic  Reasoning  (AR) 

Auto  and  Shop  Information  (AS) 

Coding  Speed  (CS) 

Electronics  Information  (El) 

General  Science  (GS) 

Mechanical  Comprehension  (MC) 

Mathematics  Knowledge  (MK) 

Numerical  Operations  (NO) 

Paragraph  Comprehension  (PC) 

Word  Knowledge  (WK) 

In  practice,  the  last  two  subtests  are  not  used 
individually,  but  rather  are  combined  into  a  single  Verbal  (VE) 
score.  Each  subtest  is  standardized  to  the  1980  youth  population 
(Department  of  Defense,  1982)  and  possesses  a  mean  of  50  and  a 
standard  deviation  of  10.  The  Armed  Forces  Qualification  Test  is 
a  composite  of  three  of  these  subtests  (i.e.,  AR  +  MK  +  2*VE) 
that  is  used  to  screen  applicants  into  the  Army.  A  second,  more 
tailored  composite  of  the  subtests  is  used  for  selection  into  a 
specific  MOS.  In  examining  the  contribution  made  by  ASVAB,  the 
individual  subtests  rather  than  the  composites  are  used  in  order 
to  maximize  the  contribution  ASVAB  can  make. 

The  new  spatial  and  psychomotor  tests  were  developed  under 
Project  A.  The  project  was  a  comprehensive  long  term  effort  to 
improve  the  Army's  selection  and  classification  system  for 
enlisted  personnel.  Specific  goals  included  developing  improved 
performance  measures  and  developing  selection  tests  which  tapped 
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a  broader  range  of  the  attribute  domain  than  already  covered  by 
ASVAB.  The  new  predictors  were  developed  through  an  iterative 
process  which  included  empirical  validation.  The  data  collected 
in  the  validation  research  was  used  to  sharpen  the  usefulness  of 
the  tests  as  well  as  our  understanding  of  the  attributes  they 
measured. 

The  new  predictors  used  in  the  present  study  can  be  divided 
into  two  groups:  spatial  and  psychomotor.  The  two  spatial 
tests,  Mazes  and  Orientation,  measured  spatial  visualization  and 
spatial  orientation,  respectively.  The  two  psychomotor  tests. 
One-  and  Two-Hand  Tracking,  measured  psychomotor  precision  and 
two-hand  coordination,  respectively.  A  short  description  of 
these  tests  along  with  the  operational  measure  used  for  each  is 
presented  in  Table  1.  More  detailed  descriptions  are  available 
in  Grafton,  Czarnolewski,  and  Smith  (1989). 


Table  1 

Description  of  Spatial  and  Psychomotor  Tests 


Test 


Target  Ability  Domain 


Measure 


Spatial 

Mazes 

Orientation 
Psvchomotor 
One-Hand  Tracking 
Two-Hand  Tracking 


Spatial  Visualization 
(Scanning) 

Spatial  Orientation 


Psychomotor  precision 
Two-hand  coordination 


Number  correct 

Number  correct 

Distance  accuracy 
Distance  accuracy 


TOW  Gunners 


Introduction 


After  enlisting  as  IIX,  Basic  Infantrymen,  and  completing 
Basic  Training  at  Fort  Benning,  Georgia,  they  were  classified 
into  MOS  IIH^,  Heavy  Anti-Armor  Weapon  Infantryman.  For  brevity 
they  are  usually  called  TOW  Gunners.  Near  the  end  of  their 
courses  in  1988,  these  students  underwent  simulator  testing  of 
their  gunnery  and  other  skills  related  to  operating  the  weapon. 


’a  recruit  enlists  with  a  "Training  MOS."  On  completing  Basic 
and  Advanced  Individual  Training,  the  recruit  is  officially 
awarded  the  MOS. 
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In  order  to  qualify  in  the  MOS,  recruits  were  required  to  exceed 
a  minimum  gunnery  score  on  the  simulator  test. 

Prior  to  TOW  (Tube-launched,  Optically-tracked,  Wire-guided) 
gunnery  training,  predictor  data  were  gathered  on  new  recruits  at 
the  Reception  Battalion.  After  gunnery  training,  gunnery 
performance  data  were  collected  on  high-fidelity  gunnery 
simulators.  This  allowed  the  predictors  to  be  validated  in  a 
predictive  design  with  relevant,  high  quality  data. 

The  TOW  anti-tank  gunnery  simulator  requires  the  gunner  to 
track  a  moving  target  (i.e.,  a  target  mounted  on  a  moving 
vehicle)  through  an  infrared  optical  device.  The  target  moves 
along  a  reasonably  level  paved  road  at  a  constant  velocity. 

Since  it  does  not  move  in  a  vertical  direction  (i.e.,  only  to  the 
degree  that  the  terrain  is  not  level) ,  these  data  may  not 
generalize  to  other  more  demanding  engagements.  The  gunner 
aligns  the  crosshair  by  swiveling  the  weapon  horizontally.  The 
qualifying  trials  were  scored  in  sets  of  10  target  engagements. 
Each  engagement  was  scored  for  accuracy  on  a  scale  of  0  to  100, 
and  the  set  score  was  simply  the  sum  of  the  scores  for  the  ten 
engagements.  A  score  of  at  least  550  on  a  set  was  required  to 
qualify.  However,  the  student  could  not  qualify  until  the  third 
set  (i.e.,  there  were  two  practice  sets).  After  attaining  the 
qualifying  score,  the  student's  testing  was  complete.  A  student 
who  did  not  attain  550  in  eight  sets  did  not  qualify  for  the  MOS. 

Data  are  presented  for  229  trainees  who  were  tested  in  1988 
on  the  new  predictors  during  their  first  three  days  of  active 
duty.  Testing  consisted  of  two  computerized  and  two  pencil-and- 
paper  tests  (Table  2) .  Scores  on  these  new  tests  were  not  used 
to  select  the  students,  who,  nevertheless,  had  been  told  before 
the  testing  that  the  scores  would  go  into  their  records. 

Two  types  of  gunnery  performance  were  measured:  1)  score  on 
first  qualifying  set,  and  2)  pass  or  fail  the  first  qualifying 
set.  The  first  measure  is  a  gunnery  accuracy  score  with  practice 
held  constant  across  trainees.  The  second  measure  is  an 
indicator  of  the  amount  of  training  needed  to  pass  the  course  and 
relates  to  the  cost  of  training. 


Results  and  Discussion 

Table  2  presents  all  validities  and  reliabilities  for  the 
ASVAB  and  spatial/psychomotor  predictors.  ASVAB  subtests  were 
related  to  score  and  pass/fail  on  the  first  qualifying  set. 
Arithmetic  Reasoning  (AR) ,  Auto  and  Shop  Information  (AS), 
General  Science  (GS) ,  and  Mechanical  Comprehension  (MC)  all 
significantly  (p  <  .05)  predicted  the  score  on  the  first 
qualifying  set.  Mechanical  Comprehension  had  the  largest 
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Table  2 

TOW  Data:  Predictor-Criterion  Relationships  for 
First  Qualifying  Set 

Criterion^ 


Pass/Fail  at  Score  at  Reliability^ 

First  First 

,  Qualifying  Qualifying 

Predictor  Set  Set 


AFQT 

★ 

.14* 

.16* 

.92 

AR 

.17 

.17* 

.  88 

AS 

.10 

.16 

.85 

CS 

.04 

.08 

.75 

El 

.03 

.10* 

.72 

GS 

.12 

.15* 

.84 

MC 

.07 

.21 

.78 

MK 

.08 

.13 

.86 

NO 

.04 

.00 

.68 

VE 

.08 

.10 

WK=.89 

PC=.72 

Soatial 

Mazes  - 

.05 

-.03 

.  70 

Orientation 

.03 

.13 

.70 

Psvchomotor 

* 

* 

.74 

One-Hand  Tracking  - 

.18* 

-.28* 

Two-Hand  Tracking  - 

.25 

-.33 

.85 

Note.  N=229  in  the 

1988 

implementation . 

*  E  <  .05. 

’positive  correlations  for  the  spatial  tests  indicate  that 
those  who  obtained  more  items  correct  were  more  likely  to  have 
passed  and  achieved  a  higher  score  at  the  First  Qualifying  Set. 
Negative  correlations  for  the  psychomotor  tests  indicate  that 
those  who  had  less  distance  from  target  were  more  likely  to  have 
passed  and  achieved  a  higher  score  at  the  First  Qualifying  Set. 

^Estimated  reliability  for  qualifying  score  (based  on 
present  data;  correlation  of  First  Qualifying  Set  with  previous 
set)  is  .63.  The  point-biserial  correlation  between  the  two 
criterion  measures  was  .73.  Of  the  229  students,  198  passed  on 
the  First  Qualifying  Set.  Mean  and  standard  deviation  of  score 
at  First  Qualifying  Set  are  636.28  and  112.59,  respectively. 

^None  of  the  predictor  reliabilities  are  based  on  present 
data.  ASVAB  reliabilities  are  alternate  forms  reliabilities  for 
Form  11a  correlated  with  Form  9a  (Palmer  et  al.,  1988).  They  are 
representative  since  most  trainees  were  tested  on  ASVAB  forms  11- 
13  and  the  reliabilities  vary  to  only  a  small  degree  across 
forms.  Since  the  reliability  of  VE  was  not  available, 
reliabilities  for  the  two  components  of  VE  (i.e.,  PC  and  WK)  are 
reported  in  its  place.  Reliabilities  for  spatial  and  psychomotor 
tests  are  test-retest  (unpublished  IPR  materials  by  J.  Toquam,  et 
al.,  PDRI,  1986). 
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correlation  with  this  criterion.  However,  for  the  prediction  of 
pass/fail  performance  on  the  first  qualifying  set,  only 
Arithmetic  Reasoning  contributed.  The  Armed  Forces  Qualification 
Test  (AFQT)  significantly  (p  <  .05)  predicted  performance  on  both 
of  these  criteria. 

Turning  to  the  new  tests ,  only  the  psychomotor  tests  ( i . e . , 
both  One-  and  Two-Hand  Tracking)  significantly  (p  <  .05) 
predicted  performance  on  both  criteria.  In  the  sample,  Two-Hand 
Tracking  fared  somewhat  better  for  both  criteria,  although  the 
differences  between  the  two  tracking  predictors  were  not 
significant  (p  >  .05).  The  cause  for  this  difference  at  the 
sample  level  may  be  that  the  Two-Hand  Tracking  test,  in  the  order 
presently  administered,  has  been  shown  to  be  more  test-retest 
reliable  than  the  One-Hand  Tracking  test  (Toquam,  Peterson, 

Rosse,  Ashworth,  Hanson,  and  Hallam,  1986) . 

A  backward  regression  approach  to  building  a  model  for  all 
criteria  involved  the  following:  1)  backward  regressing  on  only 
the  9  Armed  Services  Vocational  Aptitude  Battery  (ASVAB) 
subtests,  2)  forcing  in  the  previously  identified  ASVAB  subtests 
and  backward  regressing  on  only  the  new  spatial/psychomotor 
subtests,  3)  removing  any  ASVAB  subtests  not  significant  at  the 
.05  level  and  recalculating  the  model. 

For  both  criteria  there  was  a  significant  (p  <  .05)  amount 
of  validity  added  by  the  new  tests  above  ASVAB  subtests.  For 
details  see  Tables  3  and  4.  For  the  score  on  the  first 
qualifying  set,  Two-Hand  Tracking  and  Mazes  predicted  an 
additional  9.6  percent  of  the  criterion  variance  (i.e.,  over  a 
200  percent  increase  in  explained  criterion  variance) .  since 
Mazes  did  not,  by  itself,  significantly  predict  score  on  the 
first  qualifying  set,  its  impact  was  felt  because  it  suppressed 
variance  in  the  other  predictors.  This  is  confirmed  in  Table  5 
where  it  can  be  seen  that  Mazes  significantly  correlates  with 
both  MC  and  Two-Hand  Tracking. 

The  criterion  of  pass/fail  on  the  first  qualifying  trial  was 
also  significantly  enhanced  (p  <  .05)  by  Two-Hand  Tracking.  In 
terms  of  Somer’s  D  (i.e.,  an  index  of  rank  correlation  between 
predicted  probabilities  and  observed  outcomes) ,  adding  Two-Hand 
Tracking  to  ASVAB  raised  the  index  from  .292  to  .486,  for  an 
increase  of  .194. 

In  summary,  in  predicting  TOW  gunnery  performance  indices, 
the  usefulness  of  Two-Hand  Tracking  was  consistently  high.  Mazes 
was  the  only  other  useful  predictor  but  its  impact  was  only  as  a 
suppressor. 
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Table  3 

TOW  Data:  Predicting  Score  at  First  Qualifying  Set 


Model 

Predictor 

R 

F 

P 

ASVAB  Only 
MC 

.214 

10.95 

.0011 

ASVAB  Plus 
MC 

New  Tests 

.377 

.142 

12.44 

5.30 

<  .0001 
.0222 

Two-Hand 

Mazes 

Tracking 

-.304 

-.123 

24.26 

4.01 

<  .0001 
.0463 

Increment 

.0963 

25.17 

<  .01 

Note.  N=229.  Backward  stepwise  regression  was  used.  No 
adjustments  or  corrections  were  applied  to  the  data. 


Table  4 

TOW  Data:  Predicting  Pass/Fail  at  First  Qualifying  Set 


Models 

Concordance 

Somer 

Dyx 

P 

ASVAB  Only 
AR 

.646 

.292 

6.95 

.0084 

ASVAB  Plus 
AR 

Two -Hand 

New  Tests 

Tracking 

.743 

.486 

17.05 

.0002 

Increment 

.097 

.194 

10.10 

<  .01 

Note.  N=229.  Backward  stepwise  logistic  regression  was  used. 
No  adjustments  or  corrections  were  applied  to  the  data. 
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Table  5 

TOW  Data:  Selected  Predictor  Relationships 


MC 

AR 

Two-Hand 

Tracking 

AR 

* 

.41 

Two-Hand  Tracking 

-.27* 

* 

-.16 

Mazes 

.16* 

.11 

* 

-.24 

Note.  N=229.  *  p  <  ‘OS. 


Ml  Armor  crewman 


Introduction 


This  section  describes  the  performance  of  a  sample  of  entry 
level  trainees  at  Fort  Knox  in  MOS  19K,  Ml  Armor  Crewman,  on 
spatial/psychomotor  tests  and  school  gunnery  exercises  in  1988. 
Scores  on  the  new  spatial  and  psychomotor  tests  were  not  used  to 
select  the  trainees,  who,  nevertheless,  had  been  told  before  the 
testing  that  the  scores  would  go  in  their  records. 

Near  the  end  of  their  course,  these  students  underwent 
simulator  testing  of  their  gunnery  skills.  For  this  purpose,  the 
Unit  Conduct  of  Fire  Trainer  (UCOFT)  high-fidelity  tank  gunnery 
simulator  was  used.  The  measure  of  performance  used  in  the 
analyses  is  a  speed/accuracy  composite  constructed  by  subtracting 
the  standardized  opening  time  (i.e.,  time  from  when  a  target 
appeared  until  the  first  round  was  fired)  from  the  standardized 
percent  hits.  This  score  was  then  transformed  into  a  T-score 
with  a  mean  of  50  and  a  standard  deviation  of  10. 

Results  and  Discussion 


Table  6  presents  all  zero-order  validities  and  reliabilities 
for  the  ASVAB  and  spatial/psychomotor  predictors.  Many  of  the 
ASVAB  subtests  that  correlated  with  the  TOW  performance  indices 
also  correlated  with  the  UCOFT  speed/accuracy  performance 
composite.  The  two  largest  correlations  were  with  Auto  and  Shop 
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Table  6 

UCOFT  Data:  Predictor-Criterion  Relationships  for  Speed/Accuracy 


Predictor^ 

Speed/Accuracy^ 

Reliability^ 

AFQT 

.23: 

.92 

AR 

.21* 

.88 

AS 

.33 

.85 

CS 

.06 

.75 

El 

.03, 

.72 

GS 

.24* 

.84 

MC 

.42* 

.78 

MK 

.18 

.86 

NO 

-.06. 

.68 

VE 

.23 

WK=.89 

PC=.72 

Soatial 

Mazes 

.35: 

.70 

Orientation 

.35 

.70 

Psvchomotor 

One-Hand  Tracking 

* 

-.40* 

.74 

Two-Hand  Tracking 

-.43 

.85 

Note.  N=244  in  1988 

* 

implementation . 

*  P  <  .05. 


^Positive  correlations  for  the  spatial  tests  indicate 
that  those  who  obtained  more  items  correct  were  more  likely 
to  have  achieved  a  higher  speed/ accuracy  score.  Negative 
correlations  for  the  psychomotor  tests  indicate  that  those 
who  took  less  time  to  respond  or  had  less  distance  from  the 
target  were  more  likely  to  have  achieved  a  higher 
speed/accuracy  score. 

^Mean  and  standard  deviation  of  speed/accuracy 
criterion  are  50.54  and  9.19,  respectively. 

®None  of  the  predictor  reliabilities  are  based  on 
present  data.  ASVAB  reliabilities  are  alternate  forms 
reliabilities  for  Form  11a  correlated  with  Form  9a  (Palmer 
et  al.,  1988).  They  are  representative  since  most  trainees 
were  tested  on  ASVAB  forms  11-13  and  the  reliabilities  vary 
to  only  a  small  degree  across  forms.  Since  the  reliability 
of  VE  was  not  available,  reliabilities  for  the  two 
components  of  VE  (i.e.,  PC  and  WK)  are  reported  in  its 
place.  Reliabilities  for  spatial  and  psychomotor  tests  are 
test-retest  (unpublished  IPR  materials  by  J.  Toquam,  et  al., 
PDRI,  1986). 
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Information  and  Mechanical  Comprehension  (.33  and  .42, 
respectively) .  All  new  predictors  were  significantly  related  to 
the  speed/accuracy  score  on  UCOFT.  Mazes  and  Orientation  were 
related  to  the  criteria  to  the  same  degree  (i.e.,  0.35).  One- 
and  Two-Hand  Tracking  were  also  related  to  the  criterion  to  about 
the  same  extent  (i.e.,  -0.40  and  -0.43,  respectively;  see  Table 
8)  . 


The  backward  regression  approach  used  to  build  TOW 
prediction  models  was  also  used  for  the  UCOFT  analysis.  In 
addition  to  the  ASVAB  subtests,  both  Two-Hand  Tracking  and  Mazes 
added  validity  to  predicting  the  speed/accuracy  score  on  the 
UCOFT.  Although  Orientation  and  Mazes  had  equal  zero-order 
correlations  with  the  speed/accuracy  score.  Mazes  shared  more 
variance  with  speed/accuracy  once  both  the  ASVAB  subtests  and 
Two-Hand  Tracking  were  included  (see  Tables  7  and  8) . 

In  summary,  the  usefulness  of  Two-Hand  Tracking  was  again 
strongly  supported.  Mazes  also  added  predictive  power,  and  for 
this  criterion  it  added  it  through  a  direct  relationship  rather 
than  by  removing  non-criterion  related  variance  from  the  other 
predictors. 


General  Discussion 

The  tracking  tests  appear  extremely  useful  for  predicting 
the  gunnery  criteria  under  examination  (i.e.,  both  TOW  and 
UCOFT) .  Whether  One-  or  Two-Hand  Tracking  is  the  better 
predictor  is  unclear.  Obviously,  Two-Hand  Tracking  is  more 
strongly  related.  However,  because  the  order  of  administration 
was  held  constant  (i.e.,  One-Hand  Tracking  followed  by  Two-Hand 
Tracking) ,  it  is  not  clear  whether  Two-Hand  Tracking  is  a 
stronger  predictor  due  to  higher  reliability  arising  from 
practice  on  One-Hand  Tracking.  Strength  is  added  to  this 
argument  from  the  fact  that  reliability  was  higher  for  Two-  than 
One-Hand  Tracking.  Regardless,  tracking  ability  information  is 
very  useful  for  predicting  future  gunnery  performance. 

The  picture  for  the  two  spatial  predictors  is  not  as 
consistent.  Mazes,  for  example,  predicts  first  qualifying  TOW 
score  and  UCOFT  speed/accuracy  score  but  does  so  through 
different  mechanisms:  Once  through  suppressor  effects  and  once 
through  direct  prediction.  In  addition,  it  does  not  predict  TOW 
pass/fail  on  first  qualifying  set.  Orientation  was  not  useful 
for  predicting  TOW  performance  scores,  but  was  useful  for 
predicting  UCOFT  speed/ accuracy  score.  Even  though,  compared  to 
Mazes,  Orientation  added  less  unique  variance  once  ASVAB  was 
used,  it  nevertheless  points  to  a  relationship  that  may  be  of  use 
if  the  ASVAB 's  composition  changes. 
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Table  7 

UCOFT  Data:  Predicting  Speed/Accuracy  Score 


Model  Predictor 

R 

F 

P 

ASVAB  Only 

.448 

30.28 

< 

.0001 

MC 

.145 

27.65 

<  .0001 

AS 

.303 

6.33 

.0125 

ASVAB  Plus  New  Tests 

.553 

26.29 

< 

.0001 

MC 

.163 

9.10 

.0028 

AS 

.130 

5.83 

.0165 

Two-Hand  Tracking 

-.224 

17.31 

<  .0001 

Mazes 

.161 

8.89 

.0032 

Increment 

.105 

36.05 

< 

H 

O 

• 

Note.  N=244.  Backward  stepwise  regression  was  used.  No 
adjustments  or  corrections  were  applied  to  the  data. 


Table  8 

UCOFT  Data:  selected  Predictor  Relationships 


Two-Hand 


MC 

AS 

Tracking 

AS 

.48 

Two-Hand  Tracking 

-.41 

-.22 

Mazes 

.27 

.18 

-.36 

Note.  N=244.  All  correlations  are  significantly 
different  from  zero  at  .05  Type  I  error. 
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Regardless  of  what  new  tests  enhance  the  prediction  of 
gunnery  criteria,  the  new  tests  definitely  provide  a  means  to 
improve  the  selection  of  TOW  and  UCOFT  gunners.  Although  the 
skills  needed  to  operate  weapons  may  be  common  to  several  weapons 
systems  (e.g.,  as  tracking  skills  appears  to  be  for  these  two 
systems) ,  it  is  also  likely  that  specific  skills  will  be  needed 
for  some  weapons  and  not  others.  For  example,  evidence  exists 
that  the  present  tests  do  not  predict  performance  on  simulators 
of  the  Chapparall  and  Stinger  air  defense  weapons  (Cast  and 
Johnson,  1988) . 

When  performance  across  the  two  weapons  systems  is  predicted 
to  a  different  extent  by  the  same  tests,  it  may  be  signaling  that 
different  skills  are  needed  for  operating  the  weapons.  For 
example,  the  UCOFT  gunner  is  more  isolated  from  the  outside-the- 
tank  environment  and  thus  may  rely  more  on  the  spatial 
visualization  and  orientation  skills  measured  by  the  Mazes  and 
Orientation  tests.  In  addition  to  the  predictive  power  they 
provide,  these  specialized  ability  tests  can  also  be  used  for 
improving  classification  efficiency.  Examination  of  other 
gunnery  MOS  may  suggest  hypotheses  as  to  which  aspects  of  the 
different  weapons  systems  demand  which  skills. 
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validity  of  AFQT 
performance 


and  Education  Level  for  Predicting  Hands-On 
in  llH  TOW  Heavy  Anti-TanX  Missilemen 


Jay  M.  Silva 

This  report  was  prepared  as  a  response  to  a  technical 
retmest  frem^^the  Deputy  Chief  of  Staff  for  Resource  Management  in 
sri?ainlng  Sd  Doctrine  command.  This  office  is  conducting 
reseSIh  (Viz!,  soldier  Performance  Research  Reject)  aimed  at 

examining  the  continued  use  of 

Ratterv  TASVAB) .  Toward  this  end,  the  project  is 
slewSf tfexISni  toe  Validity  of  the  ASVAB  applied  to  a  broader 
ranqe  of  performance  measures.  Hands-on  performance  data 
recently  collected  at  Fort  Banning  provides  one  opportunity  to 

address  this  issue. 

The  purpose  of  this  report  is  to  examine  the  relationship 
between  the  Armed  Forces  Qualification  Test  (AFQT)  and 
eScSioS?  aSainment  (i.e.,  high-school  graduate  or  not  with 
performance  on  hands-on  tasks  for  TOW 
tracked  Wire-guided)  Heavy  Anti-Tank  Missilemen. 
measures  may  predict  this  performance  as  well  as  or  better  than 
AFQT  and  educational  attainment,  the  focus  here  is  solely  on 
these  two  predictors. 

The  342’  cases  for  these  analyses  took  the  ASVAB  as 
civilians  before  entering  the  Army.  After 

Basic  Infantrymen,  and  completing  Basic  [  Training  (AIT) 

Bennina  Georgia,  they  entered  Advanced  Individual  Training  (Aiij 

in  MOS  IIH^,  TOW  Gunner.  At  the  conclusion  of  ^ 

training  the  recruits  were  tested  on  seven  hands-on  tasks 
directly* related  to  their  effectiveness  as  TOW  Gunners.  The 
tlotlol  eSch  hands-on  task  is  the  number  of  tries  necessary  to 
perform  sufficiently  well  to  pass. 

Onlv  four  of  the  eight  hands-on  tasks  were  retained  for 
analySs^'since  only  those  have  variation  in 

scores  across  recruits.  The  four  hands-on  tasks  are. 


^An  additional  178  recruits  were  dropped  from  the  analysis 
aue  to  missing  data  on  one  or  more  of  the  following 
information:  AFQT  score,  high  school  diploma  status,  hand 

scores. 

»A  refcruit  enlists  with  a  "Training  MOS.”  On  completing 
Basic  Combat  Training  and  Advanced  Individual  Training,  the 
recruit  is  officially  awarded  the  MOS. 


1 


T?ts.k 


pescription 

Assemble  the  TOW  launcher 


2  Conduct  system  check  out  procedures  and  pre- 
operational  inspection  of  the  TOW  launcher  and 
encased  missile 

3  Determine  if  a  target  can  be  engaged 

4  Prepare  an  anti-armor  range  card  for  TOW 

Means,  standard  deviations,  and  maximum  n^er  of  attempts 
needed  to  pass  each  hands-on  task  are  pesented  in  Table  1.  Task 
A  f-rtriv  nn  averaae  the  most  attempts  to  complete,  and  also 
shoSS'the  of  variability.  Task  3  took  the  lowest 

number  of  attempts  to  complete  and  displayed  the  smallest  amoun 
of  variability. 

Table  1 

Moans  ar»t^  Standard  Deviations  On  Selected  TOW  pands-On  Tasks 


Hands-On  Tasks 


Mean 


Maximum 

Attempts 


Task  1  1.24  .51 

Task  2  1*19  -48 

Task  3  -b 

Task  4  1-45  *53  ^ 

Sum  of  4  tasks  4.91 _ 3.. 01 _  ^  _ 

Note.  N  was  342  for  each  task. 

The  high  school  diploma  status  was  coded  dichotomously(0=no 
diploma;  l=diploma) .  AFQT  score  is  a  percentile  score  based  on 
the  new  AFQT  composite  initiated  in  January  1989  (!•«•»  ^VE  +  AR 
;  Sk?  Table  2  lists  the  zero-order  correlations  of  the  two 
prSictoS  and  total  hands-on  performance.  Only  AFQT  was  related 
to  total  attempts  to  pass  hands-on  tasks. 
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Table  2 

yriaans  and  Standard  Deviations  of  Predj-ctQrs  .and  ^ero-Order 
rnyrelation  Matrix 

Correlation 


Mean  SD 


Total  attempts 
to  pass 

hands-on  tasks  AFQT 


High  school 
diploma  status 


.84 


.37 


-.103 


-.070 


AFQT 


52.96  20.90 


-.231 


Mote.  N  was  342  for  each  statistic.  An  asterisk  indicates 
statistical  significance  at  p  <  .05. 

Regressing  total  number  of  attempts  to  pass  the  ^ 
on  tasks  on  high  school  diploma  status  and  AFQT  score  yielded  the 

results  in  Table  3 . 

Table  3 

Pearessinn  of  Total  Attempts  to  Pass  Hands-On  Tas^^s  On  High 
«;r:hool  nialoma  Status  and  AFQT  Score 


Predictor 


Standardized 

Beta 


SE  of 
Estimate 


High  school 
diploma  status 

AFQT 


-.088 

-.225 


.053 

.053 


2.75  .0980 

18.18  <  .0001 


Note  N  was  342  for  each  task.  Negative  signs  indicate  that 
fealnees  with  higher  predictor  scores  needed  fewer  attempts  to 
pass  hands-on  tasks. 

High  school  diploma  status  was  not 
total  hands-on  performance  if  a  Type  I  error  probability  ©J  -05 
is  used  as  the  cutoff  for  significance.  AFQT  score,  however,  is 

related  to  this  performance.  In  applied  Qj^JcSr^^it^'is 

for  every  i  standard  deviation  change  in  mean  AFQT  score,  it  is 
exoected  that  the  mean  number  of  total  trials  to  complete  the 

SSdS-on  SsL  is  reduced  by  225  standard  i®^nFS;"ol*  11^" 
example, .a  one  standard  deviation  increase  in  mean  AFQT  of  iiH 
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trainees,  can  be  expected  to  lead  to  a  mean  total  trials  of  4.68 
versus  the  existing  4.91  now  needed. 

Conclusion 

AFQT  moderately  predicts  TOW  Gunner  total  hands-on 
performance.  By  itself,  high  school  diploma  status  has  a  much 
lower  relationship  with  total  hands-on  performance,  which 
strictly  speaking  is  not  significantly  different  from  zf^o. 
Finilly^  adding  high  school  diploma  status  to  AFQT  provides  non¬ 
significant  improvement. 
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Relationship  of  APQT  Category  to  TOW  Gunnery  Performance 
Jay  M.  Silva  and  Clinton  B.  Walker 

This  report  describes  the  relationship  between  Armed  Forces 
Qualification  Test  (AFQT)  scores  and  performance  on  the  M70  TOW 
anti-tank  missile  simulator  for  Infant^  trainees.  The  goal  is 
to  examine  whether  the  quality  of  soldiers,  in  terms  of  their 
AFQT  scores,  is  reflected  in  their  performance  in  a  combat  MOS. 
Although  other  measures  may  predict  TOW  gunnery  performance  as 
well  or  even  better  than  AFQT,  the  focus  here  will  be  only  on 
AFQT. 

The  1,713’  cases  for  these  analyses  took  ASVAB  as  civilians 
before  entering  the  Army.  After  enlisting  as  IIX,  Basic 
Infantrymen,  and  completing  Basic  Training  at  Fort  Banning, 
Georgia,  they  were  classified  into  advanced  training  in  MOS  IIH  , 
TOW  Gunner.  Near  the  end  of  their  courses  in  1988,  these 
students  underwent  simulator  testing  of  their  gunnery  skills.  In 
order  to  qualify  in  the  MOS,  recruits  were  required  to  achieve  a 
minimum  passing  score  on  the  simulator  test. 

The  M70  TOW  anti-tank  gunnery  simulator  requires  the  gunner 
to  track  the  missile  onto  the  target.  The  gunner  aligns  the 
crosshair  by  swiveling  the  weapon  horizontally  and  turning  two 
hand  knobs  for  elevation.  The  qualifying  trials  on  the  M70  are 
scored  in  sets  of  10  target  engagements,  a  set  being  called  an 
event.  Each  engagement  is  scored  for  accuracy  on  a  scale  of  0  to 
100,  the  event  score  being  simply  the  sum  of  the  scores  for  the 
ten  engagements.  A  score  of  at  least  550  on  an  event  is  rec^ired 
to  qualify.  Event  3  being  the  first  on  which  qualification  is 
awarded.  After  attaining  the  qualifying  score,  the  student's 
testing  is  complete.  A  student  who  does  not  attain  550  in  eight 
events  does  not  qualify  for  the  MOS. 

The  relationship  of  AFQT  to  performance  is  revealed  in 
analyses  of  three  types  of  gunnery  scores  by  AFQT  Category: 
qualifying  score,  score  on  Event  3,  and  number  of  events  needed 
to  reach  the  qualifying  level.  These  measures  represent  two 
important  aspects  of  performance,  respectively:  accuracy,  with 
practice  held  constant  across  trainees;  and  amount  of  practice 


’An  additional  47  recruits  did  not  ever  achieve  a  qualifying 
score.  The  analysis  of  these  recruits  is  presented  separately  in 
the  Appendix.  The  reason  they  were  not  included  in  the  main 
analysis  was  that  it  was  not  clear  why  they  had  not  qualified 
(e.g.,  96  percent  of  these  recruits  did  not  use  all  8  events  to 
attempt  to  qualify) .  Eighteen  cases  did  not  have  AFQT  category 
available. 

^A  recruit  enlists  with  a  "Training  MOS."  On  completing  basic 
and  advanced  individual  training,  the  recruit  is  officially 
awarded  the  MOS. 


needed  to  attain  qualification.  The  latter  measure  is  an 
indicator  of  training  costs. 

Table  1  shows  by  AFQT  category  the  means  and  standard 
deviations  of  the  qualifying  and  Event  3  scores,  statistical 
comparison  of  these  mean  scores  across  AFQT  categories  finds  no 
significant  differences  among  categories  1,  2,  and  3A.  However, 
when  categories  1,  2,  and  3A  were  collapsed  and  compared  to  a 
collapsed  composite  of  3B  and  4A,  a  significant  difference 
emerged  (p  <  .01).  For  the  qualifying  score,  the  estimated 
unweighted  mean  difference  between  these  two  groupings  of  AFQT 
categories  was  15  points  less  for  the  lower  AFQT  categories.  For 
Event  3  score  the  difference  was  36  points  less  for  the  lower 
AFQT  categories. 

The  fact  that  qualifying  scores,  compared  to  Event  3  scores, 
exhibited  less  of  a  mean  difference  between  AFQT  categories  was  a 
result  of  the  tradeoff  that  exists  between  gunnery  performance 
and  practice.  Those  in  the  lower  AFQT  categories  used  practice 
to  elevate  their  scores. 

Table  2  shows  the  percentage  of  trainees  who  qualified  in 
the  minimum  number  of  events  ( i . e . ,  Event  3 )  and  the  ^  percentage 
who  took  an  unusually  large  number  of  events  to  (^alify. 

Trainees  in  category  4A  were  less  likely  to  qualify  at  Event  3 
and  were  more  likely  to  need  at  least  5  events  to  qualify  (p  < 
.05) . 

These  data  support  the  conclusion  that  general  ability,  in 
terms  of  AFQT,  is  related,  although  to  a  small  degree,  to  the 
critical  combat  skill  of  gunnery.  Trainees  in  AFQT  categories  at 
or  below  3B  achieve  a  lower  accuracy  score  ah  the  time  of 
qualification;  and  a  larger  portion  of  trainees  in  category  4A 
take  more  time  to  qualify. 


Table  1 


TOW  Gunnery  Scores  bv  AFOT  Category 


AFQT 

Category 

N 

Qualifying 

Mean 

Score 

SD 

Event  3 

Mean 

Score 

SD 

1 

73 

654 

69 

610 

123 

2 

612 

653 

74 

627 

111 

3A 

391 

646 

69 

613 

113 

3B 

531 

637 

67 

597 

119 

4A 

106 

635 

67 

564 

155 

Table  2 

TOW  Gunnery  Qualification  Event  bv  AFOT  Category 


AFQT 

Category 

N 


Percent  Not  Percent  Needing 

Qualifying  More  Than  4 

On  Event  3  Events  to  Qualify 


1 

73 

19 

2 

612 

15 

3A 

391 

16 

3B 

531 

21 

4A 

106 

29 

7 
4 

8 
7 
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Appendix 


Forty  seven  of  the  recruits  apparently  did  not  qualify  for 
the  MOS  based  on  obtaining  a  score  of  550  or  better  on  Event  3  or 
later.  It  is  not  clear  why  these  recruits  did  not  qualify.  The 
following  tabulations  indicate  why  the  picture  is  not  clear:  1) 
45  recruits  (95  percent)  did  not  use  all  8  events  to  attempt  to 
qualify,  2)  8  recruits  (17  percent)  did  not  attempt  even  one 
event,  and  3)  of  the  ones  who  attempted  at  least  one  event,  only 
25  (64  percent)  even  attempted  to  qualify  on  more  than  3  events. 

A  descriptive  analysis  was  conducted  on  these  47  cases  in 
order  to  explore  the  relationship  that  may  exist  between  AFQT 
classification  and  TOW  gunnery  performance.  Caution  should  be 
excercised  in  interpreting  the  results  due  to  extremely  small 
sample  size  within  AFQT  category.  The  mean  highest  score 
achieved  and  mean  number  of  events  completed  in  each  AFQT 
category  is  presented  in  Table  Al. 

Table  Al 


Mean  Highest  Score  And  Number  of  Completed  Events  by  AFQT 
Category 


AFQT 

Category 

Number  Failing 
to  Qualify 

Mean  Highest 
Score  Achieved 

Mean  Events 
Completed 

1 

0 

- 

- 

2 

9 

488 

4 

3A 

12 

468 

3.6 

3B 

22 

493 

4.8 

4A 

4 

522 

5.3 

Note.  AFQT 

category  was  not 

available  for  18 

of  the 

recruits . 
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DATA  AHALTSIS  TO  SUPPORT  THE  COHHTTEE 
OH  THE 

PEBFOBMANCE  OF  MILITARY  PERSOHMEL 


Elizabeth  P.  Smith  &  Andrea  E.  Birnbaum 


The  analyses  discussed  here  with  the  attached  computer  output  were 
generated  at  the  request  of  the  National  Academy  of  Science’s  Committee  on  the 
Performance  of  Military  Personnel  for  the  Joint-Service  Job  Performance 
Measurement  Project.  An  effort  was  made  to  conform  to  the  specifics  of  the 
request  as  closely  as  possible.  However,  some  changes  were  necessary  due  to 
constraints  Imposed  by  the  data  and  by  available  resources. 

Data  Base 

The  data  were  taken  from  the  Project  A  Concurrent  Validation  Sample^  for 
those  Military  Occupational  Specialties  (MOS)  for  whom  Hands-on  Test  Scores 
were  available.  These  MOS  and  the  number  of  soldiers  in  each  were: 


Infantry  (IIB)  ^89 
-Cannon  Crewmember  (13B)  451 
Tank  Crewmember  (19E)  390 
Radio  Operator  (31C)  288 
Vehicle  Mechanic  (63B)  476 
Motor  Transport  Operator  (64C)  499 
Administrative  Specialist  (71L)  424 
Medical  Specialist  (91A)  391 
Military  Police  (95B)  590 

Total  3998 


The  variables  used  in  the  analyses  were  the  ten  Armed  Services  Vocational 
Aptitude  Battery  (ASVAB)  subtest  scores.  Armed  Forces  Qualification  Test 
(AFQT)  score,  the  MOS-specific  Aptitude  Area  Composite  (AA)  (i.e.  Service 
Composite)  score  and  the  following  Project  A  performance  total  test  scores: 

Hands-on  (all  tasks) 

Hands-on  (MOS-specific  tasks) 

School  Knowledge  (K3) 

Job  Knowledge  (K5) 

MOS-specific  Behaviorally-Anchored  Rating  Scale  (BARS) 

MOS-specific  Task  Ratings 

Reliability  values  (Recommendation  8)  for  the  Job  Knowledge  Task  Tests 
(K5),  the  Hands-on  Task  Tests  (all  tasks)  and  the  MOS-specific  Behaviorally 
Anchored  Rating  Scales  (BARS),  can  be  found  on  page  5-7  of  the  Project  A  1986 
Annual  Report  (Campbell,  1986).  Reliability  values  for  the  School  Knowledge 
Tests  (K3)  are  shown  in  Table  1  (data  provided  by  S.  R.  Schultz,  D.  B.  Kuhn, 
and  C.  B.  Walker,  personal  communication,  December,  1987). 

The  usual  standardized  scores  for  the  ASVAB,  AFQT,  and  AA  scores  were 
used.  The  Project  A  test  scores  were  converted  to  standard  T-scores 


(mean  -  50,  standard  deviation  -  10)  by  MOS  to  enable  comparisons  across  the 
same  metric.  Not  all  data  were  available  for  all  subjects  and  no  special 
attempts  were  made  to  impute  test  scores  for  subjects  on  these  measures. 

Analyses 

The  first  set  of  analyses  calculated  the  frequency  distributions,  range, 
mean,  median,  standard  deviation,  skewness,  kurtosls,  and  additional 
information  for  each  of  the  18  variables  by  MOS.  The  output  for  this  set  is 
labelled  ‘Descriptive  Statistics  by  MOS.”  These  results  respond  to 
Recommendation  6  of  the  'Proposals  for  Common  Data  Analysis  in  the  Joint- 
Service  Job  Performance  Measurement  Project.” 

The  second  set  of  analyses  determined  the  correlations  and  covariances 
among  the  18  variables  for  each  MOS.  The  output  is  labelled  'Correlations  for 
ASVAB,  AFQT,  AA,  and  Performance”  and  'Covariances  for  ASVAB,  APQT,  AA,  and 
Performance.”  These  results  respond  to  Recommendation  9a. 

In  response  to  Recommendation  9b,  first  we  plotted  each  variable  with  all 

others  by  MOS.  This  output  is  labelled  'Plot  of  - —  * ”  ^ere  ” ” 

represents  a  variable.  Second,  we  performed  polynomial  regressions  by  MOS. 
That  is,  we  regressed  each  variable  on  every  other  variable  using  the 
following  model:  Y«X+X^+X^+X^ .  The  difference  between  the  R^  obtained  for 
this  model  and  the  squared  product -moment  correlation  reflects  departure  from 
linearity.  We  did  not  compute  these  departures  from  linearity,  but  have 
provided  the  necessary  information  to  do  so.  These  printouts  are  labelled 
'Check  for  Linearity:  -  and  - '  Indicating  the  variables  involved. 
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(ARI  Tech.  Report  No.  813101).  Alexandria,  VA:  Army  Research  Institute 
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Footnote 

^An  earlier  report  on  Project  A  Concurrent  Validation  data 
[Arabian,  J.  M. ,  Vise,  L.  L.,  6  Young,  W.  (1988).  Army  research  to  link 
standards  for  enlistment  to  on- the- lob  performance:  6th  Annual  Report  to 
Congress.  (ARI  Working  Paper  RS-WP-88-2).  Alexandria,  VA:  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences]  presented  results  based  on 
analyses  which  excluded  subjects  with  missing  scores  on  certain  criterion 
variables .  Such  an  exclusionary  rule  was  required  for  that  report  because  of 
the  manner  in  which  criterion  construct  scores  were  generated.  The  analyses 
examined  here  did  not  require  the  exclusion  of  such  subjects.  Accordingly, 
these  results  are  based  on  different  sample  sizes  from  those  reported  in 
Arabian,  et  al.  (1988). 


Tablt  1 

iftsults  Froa  JRKTs  Adninistered  During  Concurrent  Validation 


nos 

Number 

of 

Sub.iects 

Number 

of 

Items 

Mean 

Percent 

Correct 

_S1L 

Ranoe 

Aloha 

Mean 

Number 

Correct 

Batch  A 

IIB 

491 

150 

62.1 

11.4 

93 

.93 

93.2 

13B-SP 

371 

136 

57.0 

10.1 

72 

.88 

77.5 

13B-T 

86 

156 

54.8 

11.3 

96 

.89 

85.5 

19E 

394 

162 

68.7 

10.8 

110 

.93 

111.3 

31C 

289 

175 

60.3 

11.4 

no 

.93 

105.6 

63B 

478 

171 

61.0 

11.3 

.  94 

.93 

104.4 

64C 

507 

128 

63.1 

10.6 

78 

.90 

80.8 

71L 

427 

105 

59.2 

10.7 

65 

.86 

62.2 

91A 

392 

175 

60.3 

10.3 

103 

.92 

105.6 

95B 

597 

139 

59.4 

9.6 

79 

.87 

82.5 

Batch  Z 

12B 

544 

176 

58.9 

11.5 

114 

.93 

103.7 

16S 

338 

149 

70.5 

9.0 

82 

.89 

105.0 

27E 

123 

175 

56.8 

13.6 

102 

.94 

99.4 

51B 

69 

163 

62.8 

13.6 

108 

.94 

102.4 

54E 

340 

145 

68.7 

10.3 

85 

.90 

99.5 

S5B 

203 

181 

60.1 

13.3 

122 

.95 

108.9 

67N 

238 

175 

66.3 

8.3 

81 

.89 

116.0 

76W 

339 

175 

49.8 

9.8 

95 

.90 

87.3 

76Y 

444 

175 

61.9 

10.9 

119 

.93 

108.4 

94B 

368 

175 

57.3 

10.3 

99 

.92 

100.2 
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.  ARI's  Nominations  of  New  Tests 

For  the  Technical  Advisory  Selection  Panel  (TASP) 


Clinton  B.  Walker,  Editor 


General  statement 

This  report  forwards  to  TASP  ARI's  nominations  of  new  tests 
for  use  in  the  ECAT  cost/benefit  project.  ARI  has  in  its 
databases  the  information  to  address  almost  all  of  the  issues  in 
TASP's  criteria  for  future  tests.  However,  a  number  of  useful 
analyses  will  not  have  been  completed  before  the  TASP  proposal 
on  new  tests  has  to  go  forward. 

So  far  under  Project  A,  analyses  of  Army's  new  ability  tests 
have  been  done  at  the  level  of  composites  of  predictors,  not  at 
the  level  of  the  individual  test.  The  data  from  Project  A  are 
archived  and  await  analysis  under  a  follow-on  contract  called 
Manning  the  Career  Force.  Bids  have  been  received  on  it,  and  a 
startup  by  October  1989  is  expected.  By  the  time  that  the  ECAT 
cost/benefit  project  is  ready  for  evaluation,  the  relevant  Army 
analyses  of  Project  A  data  will  have  been  completed.  These 
analyses  will  cover  fairness,  alternative  scoring  methods  (e.g., 
trading  off  speed  and  accuracy  to  defeat  testwise  response 
strategies),  and  other  items  in  the  TASP  evaluation  criteria. 

Here  is  a  brief  description  of  the  data  that  ARI  has 
archived  from  Project  A  and  that  will  support  the  relevant 
analyses.  Most  of  Army's  performance  measures  for  JPM  have  been 
collected.  Project  A  conducted  its  Concurrent  Validation  (CV)  in 
1985,  when  over  9,000  first  tour  soldiers  in  19  MOS  took  all  of 
the  new  predictors.  These  MOS  were  picked  with  the  advice  of  a 
panel  of  General  Officer  advisors  to  be  representative  and 
critical.  The  result  was  a  set  of  MOS  that  account  for  45%  of 
Army's  annual  accessions. 

Nine  of  the  MOS  in  the  CV,  these  supplying  over  5,000  cases, 
were  given  hands-on  tests  of  MOS-specific  and  general  soldiering 
tasks.  Validities  have  been  computed  so  far  for  constructs  or 
composites  like  "Spatial,"  "Computerized  Tests,"  and  "Hands  on 
Performace."  Incremental  validities  relative  to  pre-enlistment 
ASVAB  were  computed  for  these  predictor  composites.  In  those 
analyses,  the  ASVAB  score  was  a  linear  composite  of  four  factors: 
Technical,  Quantitative,  Verbal  and  Speed. 

The  next  phase  of  Project  A  was  the  Longitudinal  Validation 
(LV) ,  in  which  50,000  new  recruits  took  the  new  predictors  and 
then  had  their  success  in  Service  tested  at  two  subsequent 
points;  end  of  entry-level  job  training  (knowledge  tests  for  all 
21  MOS;  n  >40,000)  and  first  tour  on  the  job  (hands-on  and 
knowledge  tests  for  9  MOS,  total  n  about  6,000,  and  knowledge 
tests  for  11  other  MOS,  total  n  about  4,000). 


»  ' 

In  addition  to  data  from  Project  A,  ARI  has  results  from 
smaller  projects  that  have  been  carried  out  with  trainees  in  tank 
and  anti-tank  gunnery.  For  these  smaller  projects,  some  analyses 
at  the  level  of  the  individual  test  have  been  done  and  are 
summarized  in  this  report.  In  regression  analyses  of  these 
gunnery  data,  some  tests  give  inconsistent  results.  This 
variation  may  be  due  to  changes  in  the  set  of  variables  in  the 
analyses,  changes  in  the  particular  test  score  used  (e.g.,  speed 
or  accuracy) ,  changes  in  the  conditions  of  predictor  or 
performance  testing,  or  to  variations  in  samples  of  examinees. 

So  these  analyses  are  exploratory. 

On  a  scale  of  task  criticality,  hitting  targets  is  near  the 
top;  so  the  value  of  these  smaller  projects  for  TASP  is  their 
measures  of  gunnery  performance  from  high-fidelity  simulators ^ of 
weapon  systems.  Data  from  a  new  implementation  of  three  spatial 
tests  will  be  available  by  the  end  of  June.  These  tests  are 
being  piloted  as  predictors  of  success  in  land  navigation,  a  task 
that  is  important  in  many  Army  jobs. 

Incremental  validity  for  selection 

Although  TASP  is  looking  at  new  tests  mainly  for  classifi¬ 
cation,  the  minutes  to  January’s  meeting  note  that  gains  in  over¬ 
all  validity  should  be  assessed  as  well.  It  is  conceivable  that 
even  small  gains  in  pan— occupational  validity  could  enlarge  the 
expected  utility  of  new  tests  enough  to  win  continued  support 
for  them.  In  this  regard,  the  CV  has  found  overall  gains  in 
predicting  the  two  constructs  of  maximal  performance,  MOS- 
specific  Technical  Proficiency  and  General  Soldiering 
Proficiency.  Both  of  those  outcomes  are  composites  of  training 
knowledge,  on-the-job  knowledge,  and  on-the-job  hands-on  skills. 

In  the  CV,  the  average  R  for  all  ASVAB  subtests  against  MOS- 
specific  Technical  Proficiency  was  .63.  Adding  the  new 
computerized  tests  to  the  ASVAB  subtests  gains  a  point;  adding 
the  six  new  spatial  tests  gains  two  points.  For  the  outcome  of 
General  Soldiering  Proficiency,  the  R  for  all  of  ASVAB  was  .65. 

Spatial  tests  added  three  points  to  that,  computerized  tests  two 
points.  Against  the  hands-on  component  of  job-performance,  ASVAB 
had  a  validity  of  .47  compared  with  .49  each  for  ASVAB  plus  the 
spatial  and  computerized  predictors.  Other  combinations  of  the 
predictors  need  to  be  examined.  For  all  analyses  reported  in 
this  paragraph,  a  multivariate  correction  for  restriction  of 
range  was  applied  to  ASVAB  and  a  correction  for  shrinkage  used. 

The  criterion  problem 

Predictor  tests  are  not  good  candidates  for  incrementing 
validity  by  themselves;  their  potential  benefits  are  with  respect 
to  specific  performance  measures  or  classes  of  such.  In  order 
for  the  new  tests  in  the  ECAT  cost/benefit  project  to  receive  a 
fair  tryout,  they  need  to  be  validated  against  appropriate 
criteria.  This  concern  is  prompted  in  part  by  the  move  to  use 
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end  of  training  grades  as  criteria  for  a  major  part  of  the  ECAT 
cost/benefit  project.  Those  grades  come  mostly  from  knowledge 
tests,  which  we  already  know  are  well  predicted  by  ASVAB.  A  plan 
to  rely  heavily  on  end  of  training  grades  is  a  plan  not  to  look 
very  hard  for  incremental  validity. 

In  the  same  vein,  ARI  has  found  strong  relations  between  new 
Project  A  tests  and  performance  on  two  weapon  systems  (Grafton, 
et  al.,  1989;  Graham,  1988;  Smith  &  Graham,  1987;  Smith  and  M. 
Walker,  1988),  but  poor  validities  for  some  of  the  same 
predictors  against  performance  on  a  different  weapon  system  (Gast 
&  Johnson,  1988) .  Another  difference  is  between  validities  for 
predicting  performance  of  Tank  Crewmen:  when  the  criterion  is  a 
composite  of  hands-on  tasks,  the  entire  new  Project  A  battery 
(i.e.,  spatial,  computerized,  and  non-cognitive)  correlates  .43, 
but  when  the  criterion  is  tank  gunnery  alone,  the  correlation  of 
two  spatial  and  two  computerized  predictors  is  .54.  So  if  the 
fate  of  the  future  tests  rests  on  their  showing  good  incremental 
validities  in  the  ECAT  cost/benefit  project,  the  choice  of 
performance  measures  is  critical. 

A  relatively  low  cost  tactic  for  getting  objective  data  on 
MOS-specific  hands-on  tasks  is  to  include  occupations  that 
routinely  qualify  new  trainees  and/or  requalify  incumbents  on 
high-fidelity  simulators.  In  two  such  occupations,  ARI  has  found 
sizable  increments  in  validity  against  simulated  gunnery  perform¬ 
ance.  The  cost  of  putting  part-time  contract  employees  in  the 
field  at  several  posts  to  collect  routinely  generated  performance 
measures  might  be  small  compared  to  the  benefit  and  to  the 
alternative  tactics  for  getting  criterion  data. 

One  uncertainty  that  TASP  will  have  to  live  with  is  the 
life  expectancy  of  high-fidelity  simulators.  Budget  cuts  could 
curtail  routine  performance  testing  on  these  devices  at  the  end 
of  training. 

A  related  issue 

How  does  one  interpret  the  data  from  high-fidelity 
simulators?  There  is  some  concern  that  validity  for  predicting  a 
specific,  highly  critical  measure,  such  as  accuracy  in  gunnery, 
may  not  represent  validity  against  the  totality  of  performance  on 
the  job.  That  concern  is  legitimate,  but  a  reverse  concern  is 
equally  important:  performance  on  a  set  of  tasks  that  has  been 
composed  for  randomness,  convenience,  or  criticality  from  (e.g.) 
a  gunner's  job  will  not  tell  how  well  the  gunner  can  shoot  if 
shooting  is  not  one  of  the  tasks.  Since  classification  is  our 
main  concern,  we  may  be  justified  in  overlooking  randomness, 
breadth,  etc. ,  in  our  task  sampling.  Looking  at  a  few  MOS- 
unique,  highly  critical  tasks  may  be  sufficient  for  some  MOS. 
ASVAB  can  take  care  of  the  gunner's  knowledge  and  general  skill; 
the  new  tests  can  tell  how  well  the  gunner  can  hit  the  target. 
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utility  of  gains  in  classification 

An  item  that  may  need  early  discussion  is  how  to  make  a  case 
for  the  utility  of  a  test's  gains  in  classification.  Classifi¬ 
cation  may  be  justified  by  psychometric  evidence  or  by  evidence 
that  the  net  utility  of  recruiting,  training,  and  success/ 
failure  in  some  jobs  is  greater  than  in  others.  Meanwhile,  each 
Service  puts  some  constraints  on  classification  in  order  to 
distribute  able  persons  across  jobs.  These  constraints  are  meant 
to  insure  that  all  occupations  can  provide  themselves  able 
leaders  from  within.  But  constraints  on  distribution  limit  the 
impact  of  a  test  on  classification,  and  thus  the  possible  gains 
in  utility.  The  challenge  then  is  to  find  out  whether  there  will 
be  a  net  gain  in  utility  through  improved  classification  in  spite 
of  these  constraints.  Information  on  Services'  systems  for 
distributing  quality  personnel  may  be  needed. 

Organization  of  Army  entrv-level  training 

Recruits  in  many  occupations  enter  the  Army  at  one  training 
post,  take  Basic  Combat  Training  there,  and  then  go  to  a 
different  post  for  training  in  their  MOS.  If  the  ECAT 
validation  is  designed  to  administer  predictors  at  RTCs/Reception 
Battalions  and  to  have  the  end  of  training  as  one  source  of 
performance  measures,  then  the  two  kinds  of  data  may  have  to 
come  from  different  posts  for  some  Army  occupations.  In  the  case 
of  IIH  TOW  Gunners  and  19K  Tank  Crewmen,  students  take  their 
basic  and  technical  training  at  the  same  post.  But  for  the  other 
MOS  that  Army  is  offering,  13F  Fire  Support  Specialist,  students 
entering  technical  training  at  Fort  Sill  come  from  several  basic 
training  posts. 

The  more  new  tests  the  better 

Since  the  time  for  predictor  testing  in  the  ECAT  cost/ 
benefit  project  is  limited,  TASP  may  not  be  able  to  incplude  all 
of  the  new  tests  that  show  promise.  One  block  of  testing  time 
that  bears  examining  is  the  one  for  CAT-ASVAB.  ARI  believes  that 
there  will  be  a  much  better  payoff  from  the  cost/benefit  project 
by  devoting  all  the  testing  time  to  new  tests  than  by  dropping 
new  tests  so  that  CAT-ASVAB  can  be  administered  concurrently. 
Here's  why; 

A  close  facsimile  of  concurrent  CAT-ASVAB  scores  will  be 
available  for  each  examinee;  pre-enlistment,  pencil/paper  ASVAB 
scores.  For  any  new  test  that  is  excluded  from  the  ECAT  battery 
there  will  be  no  joint-Service  facsimile.  Moreover,  CAT-ASVAB 
will  have  another  opportunity  to  interact  with  new  tests  in  the 
large  scale  data  collection  that  would  be  needed  to  bring  new 
tests  into  the  ASVAB.  But  any  new  test  that  is  dropped  now  will 
not  have  another  chance  to  compete  for  a  place  in  Joint  Service 
pre-enlistment  testing  for  a  very  long  time.  The  statistical 
interplay  of  the  new  tests  (e.g.,  redundancies,  suppressor 
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effects)  is  as  important  as  their  individual  validities,  so  it  is 
essential  to  have  as  many  of  them  as  possible  included  now. 
Irrelevant  response  strategies 

A  number  of  tests  are  proposed  in  this  paper  because  they 
will  help  identify  examinees'  special  abilities  for  tasks  in 
combat  MOS.  Those  MOS  are  hard  to  fill  owing  partly  to  their 
large  size  and  partly  to  the  lack  of  equivalent  civilian  jobs. 
Applicants  could  well  try  to  steer  themselves  away  from  combat 
MOS  and  toward  MOS  with  civilian  counterparts  by  doing  poorly  on, 
e.g.,  psychomotor  tests.  In  this  time  of  the  all  volunteer 
forces,  we  usually  do  not  expect  examinees  to  "fake  bad,"  but 
such  faking  needs  to  be  added  to  the  research  agenda  for  the  new 
tests . 
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TESTS  OF  SPATIAL  ABILITIES  (pp.  8  -  37) 

Assembling  Objects  Map 

Mazes  Reasoning 

Orientation 

This  section  of  the  report  describes  five  of  the  six  spatial 
tests  from  the  Project  A  battery.  Validities  of  the  individual 
tests  against  the  Project  A  performance  constructs  have  not  been 
computed  yet.  In  the  table  below  are  validities  and  incremental 
validities  for  a  unit  weighted  composite  of  all  six  spatial  tests 
against  a  composite  of  MOS-specific  hands-on  tasks  in  Project  A. 

Project  A  Concurrent  Validation 
Validities  Against  Hands-on  Performance  in  Nine  Major  MOS 
For  ASVAB  and  Spatial  Predictor  Composites 

Occupations 


Predictor 

Composites 

IIB 

13B 

19E 

3 1C 

63B 

64C 

71L 

91A 

95B 

ALL 

ASVAB  (4  fac¬ 
tors,  unit  wtd) 

.44 

.22 

.40 

.65 

.30 

.54 

in 

• 

.58 

.59 

.47 

All  Spatial 
(unit  wtd) 
Spatial  + 

.39 

.24 

.40 

.51 

.26 

.53 

.53 

.55 

.57 

.44 

ASVAB 

.45 

.24 

.43 

.64 

.33 

.57 

.58 

.60 

.61 

.49 

Source:  McHenry,  Szenas,  &  Wise  (1987,  unpublished) 


Names  of  the  occupations 

IIB  Infantryman 
13 B  Cannon  crewman 
19E  Armor  crewman 
3 1C  Radio  teletype  operator 
63B  Light-wheel  vehicle 
mechanic 


are  as  follows: 

64C  Motor  transport 
operator 

71L  Administrative 
specialist 

91A  Medical  specialist 
95B  Military  police 


It  now  appears  that  the  performance  measures  for  validating 
new  tests  in  ECAT  will  be  solely  end-of-training  knowledge  tests, 
at  least  in  some  occupations.  An  indication  of  the  power  of 
Army's  new  spatial  tests  to  predict  scores  on  knowledge  tests 
(qua  pencil/paper  multiple  choice  tests)  is  given  in  the  next 
table.  There,  a  composite  of  end-of-training  knowledge  tests  and 
on-the-job  knowledge  tests  in  the  Project  A  CV  is  the  criterion. 

Unlike  end-of-training  tests  that  are  a  part  of  training 
courses,  the  Project  A  knowledge  tests  were  designed  to  have 
ample  variances.  Thus,  the  correlations  in  the  following  table 
are  probably  an  optimistic  estimate  of  the  validities  to  be 
expected  in  the  ECAT  validation. 
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Project  A  Concurrent  Validation 
Validities  Against  Knowledge  Tests  in  Nine  Major  MOS 
For  ASVAB  and  Spatial  Predictor  Composites 


Predictor 

Composites 

IIB 

13B 

19E 

ASVAB  (4  fac¬ 
tors,  unit  wtd) 

.77 

.61 

.76 

All  Spatial 
(unit  wtd) 
Spatial  + 

.70 

.62 

.72 

ASVAB 

.80 

.66 

.79 

31C 

63B 

64C 

71L 

91A 

95B 

ALL 

.77 

.80 

.74 

.73 

.81 

.78 

.75 

.66 

.70 

.67 

.68 

.70 

.69 

.68 

.77 

.81 

.75 

.76 

.82 

.80 

.77 
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Outline  of  case  for  Assembling  Objects  to  be  in  ECAT  Validation 


Description:  Assembling  Objects  is  a  somewhat  speeded  36  item 

test  from  the  spatial  battery  of  Project  A.  The 
item  "stem”  is  a  figure  showing  a  set  of  separate 
parts.  Examinees  select  from  multiple  choices  the 
one  that  is  a  possible  assemblage  of  the  parts. 

The  parts  assemble  into  a  regular  geometrical 
outer  border  or  into  a  tinker-toy  like  design. 
Directions  and  sample  items  are  given  below. 


ASSEMBUNG  OBJECTS  TEST 


In  this  tstt  you  art  to  figurt  out  how  an  objact  will  look  whan  its  parts  ara  put  togtthar  corractly. 

For  aach  problem,  the  first  picture  shows  the  parts  that  need  to  be  put  together.  The  next  four  pictures,  labeled 
A,  B,  C.  and  0,  show  four  different  ways  the  parts  could  be  put  together.  Only  one  of  them  is  correct. 

There  are  two  types  of  problems  in  the  test.  In  the  first  type,  the  parts  are  labeled  with  letters,  and  by  matching 
the  letters,  you  can  see  where  the  parts  should  touch  when  the  object  is  put  together  correctly.  Look  at 
Example  1. 


Example  1: 


□ 

0  0#© 


First,  look  at  the  square  with  the  corner  marked  “a*‘.  Imagine  what  the  object  would  look  like  if  you  attached 
that  corner  to  the  end  of  the  line  marked  "a",  and  then  attached  the  "b"  end  of  the  line  to  the  middle  of  the 
circle,  where  the  "b*’  is.  Of  the  four  answer  choices,  labeled  A,  B.  C«  and  0.  only  C  shows  the  object  put 
together  correctly,  so  circle  C  has  been  filled  in. 

For  the  second  type  of  problem  in  this  test,  the  pans  to  be  put  together  are  not  labeled,  in  these  problems,  you 
just  fit  the  parts  together  like  the  pieces  of  a  puzile.  Only  one  of  the  four  answer  choices  is  correct.  Look  at 
Example  2  below. 

Example  2: 


When  you  piece  together  the  parts  in  the  first  picture,  only  answer  choice  D  can  be  correct.  D  is  the  only 
answer  choice  that  contains  all  the  right  shapes  or  pieces  of  the  puzzle. 

To  mark  your  answer,  fill  in  the  lettered  circle  below  the  correct  answer  choice,  as  shown  in  the  tvyo  examples 
above. 
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1.  Theoretical  development  of  construct  and  measure (s) 

A.  Definition  of  construct  (process [es] ) 

Domain:  Spatial  abilities 

Sub-domain:  Spatial  visualization  (Rotation) 

The  ability  to  restructure  pieces  of  a  two-dimensional 
display  by  mentally  manipulating  them  and  judging  the 
results.  Rotation,  here  two  dimensional,  is  the  more 
specific  ability  to  identify  a  two-dimensional  figure 
in  different  orientations  in  the  plane  of  the  page. 

How  is  this  construct  related  to  other  constructs/ 
factors?  In  Alderton's  discussion  of  relations  among 
spatial  tests,  he  places  the  Minnesota  Paper  Form  Board, 
a  test  whose  items  resemble  the  bounded  figure  items  of 
Assembling  Objects,  at  the  "complex"  and  "power"  ends  of 
the  dimensions  of  complexity  and  speed  vs.  power.  For 
correlational  data,  see  below  3. A.  Construct  Validity. 

B.  Taxonomy/types/categories  of  items 

Two  kinds  of  items  are  on  this  test.  In  the  first,  the 
parts  of  the  object  are  all  closed  figures  and  the  object 
that  they  form  is  a  recognizable  geometrical  shape  (e.g., 
circle  or  polygon) .  In  the  second  type,  the  parts 
include  lines  and  angles  along  with  small  closed 
figures.  Response  choices  then  look  like  small  tinker- 
toy  assemblages,  or  abstract  mobiles.  In  the  stem,  the 
parts  are  labelled  with  letters  indicating  where  they 
connect  with  the  correspondingly  lettered  components  of 
other  parts.  The  letters  define  and  limit  correct 
arrangements  of  the  parts  together. 

How  the  test  is  scored/scaled:  Total  number  correct 

How  are  item  type  and  difficulty  jointly  sampled: 

Eighteen  of  each  type  are  on  the  test.  Examination  of 
design  and  statistical  parameters  for  the  items  is 
needed  to  finish  answering  this  question. 

C.  Seeming  differences  across  subgroups  in  e^eriences 
relevant  to  any  type  of  item  or  in  offensiveness  of 
content  (e.g.,  experience  with  computers):  Likely  SES 
related  differences  in  exposure  to  abstract  diagrammatic 
stimuli. 

D.  Basis  for  expecting  incremental  validity  in  specific 

jobs:  "Predictive  validity  estimates  provided  by  expert 

raters  [Wing,  Peterson,  and  Hoffman,  1984]  suggest  that 
measures  of  the  visualization  construct  would  be 
effective  predictors  of  success  in  MOS  that  involve 
mechanical  operations  (e.g.,  inspect  and  troubleshoot 
mechanical  [or]  electrical  systems) ,  construction. . . 

and  drawing  or  using  maps.  (p.  3-5)"  (Peterson,  1987) 
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Alderton  (1989)  reports  that  the  Minnesota  Paper  Form 
Board,  which  has  items  (juite  like  the  second  item  type 
on  AO,  has  produced  significant  validities  against  grades 
in  courses  for  following  jobs:  aircraft  inspector,  air 
traffic  controller,  navigator,  sewing  machine  operator, 
pharmaceutical  packer,  auto  mechanic,  carpenter,  gun¬ 
boat  and  motor-launch  operator,  signal [person] ,  and 
plumber.  These  jobs  either  resemble  or  appear  to  have 
important  elements  in  common  with  enlisted  Army  jobs. 

2.  Precision  of  measurement 

A.  Test's  consistency  across 
Different  forms:  One  form  exists  now. 

Testing  occasions:  Retest  reliability  was  .70  for  >  468 
Army  incumbents.  In  67  Marines,  the  value  was  .51 
(source:  4/88  DAC  briefing) . 

Levels  of  total  test  score  (score  conditional  SEM) : 

To  be  determined  under  the  contract  that  follows  up 
Project  A. 

B.  For  power  or  speeded  test:  internal  consistency: 

Alpha  of  .90  in  9,000+  incumbents;  .91  Split  half,  with 
Spearman-Brown  correction) .  In  the  Marine  sample  (n  = 
67),  .89. 

C.  Variation  across  subgroups  in  score-conditional  SEM: 

To  be  determined  under  the  contract  that  follows  up 
Project  A. 

D.  If  score  is  model-based,  what's  the  model-based 

precision:  N.A. 

E.  Subgroup  differences  in  item  functioning: 

Fairness  analyses  on  this  test  will  be  done  under  the 
follow-on  contract  to  Project  A.  Group  means  and 
standard  deviations  in  the  Concurrent  Validation  were  as 
follows: 


Group  n 

X 

s.d. 

Whites  6,007 

24.75 

6.04 

Blacks  2,551 

19.79 

6.99 

Hispanics  334 

23.87 

6.19 

Other  355 

23.97 

6.27 

Females  887 

23.44 

6.33 

Males  8,360 

23.31 

6.73 

3.  Validity 

A.  Construct  validity:  correlations  with  other  tests  of 
similar  things.  Intercorr.  with  ASVAB:  In  the  field 
test  of  the  Project  A  predictor  battery.  Assembling 
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Objects  and  a  marker  test  for  the  construct,  FIT 
Assembly,  had  a  correlation  of  .64  in  56  examinees. 

Alderton  (1989)  supports  his  position  that  the  Minnesota 
Paper  Form  Board  is  a  power  test  of  high  complexity  by 
showing  that  its  intercorral t ions  with  other  tests  of 
varying  speededness  and  complexity  are  as  they  should  be. 
The  second  item  type  in  AO  resembles  those  on  the  MPFB. 

Assembling  Objects  is  the  test  that  loads  most  heavily  on 
the  spatial  factor  among  Project  A  tests  and  ASVAB 
subtests  (p.  81) .  In  addition,  it  has  no  high  loadings 
on  other  factors  in  the  analyses  of  pre-enlistment  ASVAB 
and  the  Project  A  battery.  Among  ASVAB  subtests,  its 
highest  correlations  are  with  MC,  AR,  and  MK  (p.  86) . 

B.  Uniqueness  re  ASVAB.  How  assessed?  Using  internal 
consistency  (.91)  or  retest  reliability  (.70)  minus 
the  squared  multiple  R  of  .26  with  all  ASVAB  subtests, 
uniqueness  is  .65  or  .44,  respectively.  Paul  Mayberry's 
April  88  briefing  to  DAC  reported  somewhat  lower 
reliabilities  and  uniqueness  based  on  67  cases. 

C.  Incremental  validity  re  ASVAB?  Improvement  in  SD  of  the 

predicted  criterion?  How  many  other  tests  were 
considered  when  this  one  was  validated?  Incremental 

validities  of  all  computerized  and  spatial  predictors 
from  Project  A  were  examined  in  GLM  analyses  by  Silva 
(1989,  unpublished).  The  sample  was  311  trainees  who 
had  taken  ASVAB  as  applicants.  Project  A  tests  during 
their  first  three  days  of  Service,  and  tests  of  anti-tank 
gunnery  skills  about  twelve  weeks  later.  The  criterion 
measure  was  an  accuracy  score  on  the  fifth  event  (i.e., 
set  of  10  trials) ,  which  was  the  first  event  on  which  a 
gunner  could  attain  qualification.  Correlations  of  all 
Project  A  tests,  AFQT,  and  ASVAB 's  Combat  Composite  with 
the  criterion  were  first  examined.  Variance  in  the 
criterion  accounted  for  by  two  sets  of  tests  -  all 
Project  A  and  all  ASVAB  -  was  computed  separately  and 
then  same  analysis  was  run  on  the  Project  A  tests  with 
the  criterion  variance  associated  with  ASVAB  removed. 
Finally,  the  Project  A  tests  were  entered  into  a 
stepwise  linear  regression  on  the  residual  (unrelated  to 
ASVAB)  variance. 

AO  correlated  .082  with  Event  5  scores  (p  =  .1305)  and 
was  not  significantly  associated  with  unique  criterion 
variance  in  analyses  that  include  or  excluded  ASVAB 
subtests  and  ASVAB-related  variance  (p  =  .082  in  the 
latter  case) .  The  multiple  R^  for  all  ASVAB  subtests  was 
.041,  for  all  Project  A  was  .166,  and  for  both  sets 
together  .177.  Project  A  tests  accounted  for  .126  of  the 
residual  variance.  In  the  stepwise  regression  (p  =  .15 
for  entry)  for  predicting  residual  variance,  AO  is  the 
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fourth  of  five  Project  A  tests  to  enter  (p.  100) . 

The  Marines  have  found  in  JPM  work  to  date  that 
Assembling  Objects  adds  .061  in  explained  variance  on  top 
of  four  ASVAB  factors,  MOS,  time  in  service,  and  time  in 
service  squared  against  a  knowledge  test,  in  a  sample  of 
>1,100  Marines  (unpublished  data  reported  to  JPM  5/89). 

In  addition,  it  adds  .032  to  the  explained  variance  in 
predicting  hands-on  performance.  Among  the  predictors  in 
the  Marines'  battery  were  a  couple  of  psychomotor 
measures,  "spatial,"  Project  A's  Reasoning  test,  and  the 
Armed  Services  Applicant  Profile. 

D.  Subgroup  differences  in  incremental  validity  or  in  std 
err  of  prediction  of  the  criterion?  To  be  determined 
under  the  follow  on  contract  to  Project  A. 

E.  Job  families  and  patterns  of  validities  for  them. 
Similarity  of  job  families  in  other  Services. 

Expected  utility  in  terms  of  numbers,  attrition,  training 
costs,  attractiveness  to  applicants:  For  incremental 
validities  of  a  composite  of  Project  A  spatial  tests 
against  major  outcomes  in  nine  MOS,  see  above  pp.  6  -  7. 

4 .  Equating 

A.  Problems  in  developing  equivalent  forms  (e.g. ,  How  large 
is  the  potential  item  pool?)  Would  pretesting  and 
vetting  of  new  items  be  needed?  Could  item  devel  be 
automated?  The  potential  item  pool  is  unlimited.  Over 
the  short  run,  additional  items  could  be  created  by 
rotating  whole  items  or  squares  of  information  within 
items.  Pretesting  of  new  items  would  be  advisable. 

Item  development  probably  could  be  automated,  but  not 
cheaply. 

B.  Types  of  scunples  needed/acceptable  for  initial  equating? 
testing  conditions  needed?  Sufficiency  in  cells  of 
minority,  female,  and  low  aptitude  examinees  would  drive 
sample  sizes 

C.  Subgroup  dependence  of  equatings:  To  be  determined  at 
the  point  where  we  get  into  equating. 

D.  Practicality  of  pencil-and-paper  versions:  Assembling 
objects  started  as  a  pencil-and-paper  test. 

5.  Feasibility  for  operational  S/C: 

A.  Min. /Max.  admin  times.  Times  for  instrux/test  for  low 
and  high  AFQT  examinees:  There  is  a  fixed  time  for 
answering  the  items  of  16  minutes.  Time  for  giving 
directions  is  about  5  minutes.  Dave  Alderton  (personal 
communication,  1989)  reports  the  following  time  data 
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from  administering  a  computerized  AO  to  1,305  Navy 
recruits.  Navy's  software  permitted  examinees  to  skip 
questions.  Skipped  items  were  automatically  presented 
again  at  the  end  of  the  test,  if  time  allowed. 

Component\  mean _ s.d.  maximum  time 

Instructions  4.75  min  1.46  14.85 

Items  10.66  1.99  12.22 

Total  15.41  2.82  26.85 

B.  Susceptibility  to: 

Cheating,  compromise:  Estimated  to  be  low:  verbal 
descriptions  of  the  items  probably  would  not  be  easy 
to  memorize. 

Irrelevant  response  strategies:  On  their  face,  the  items 
seem  to  invite  using  a  verbal  response  strategy.  The 
findings  that  Assembling  Objects  anchors  the  spatial 
factor  in  the  Concurrent  Validation  and  that  it  adds  a 
useful  amount  of  variance  on  top  of  ASVAB  in  Marine 
research  show  that  verbal  strategies  do  not  automatically 
wash  out  the  test's  unique  contribution. 

Coaching:  Unknown;  guessed  to  be  low 

Practice:  An  earlier  version  of  Assembling  Objects  which 
had  four  more  items  showed  a  gain  .08  s.d.  in  502 
incumbents  (p.  97) . 

C.  Floor/ceiling  effects?  Performance  of  low/high  AFQTs  on 
the  test?  On  retesting,  the  mean  score  of  23.96  was  1.8 
s.d.  below  a  perfect  score.  Performance  of  different 
AFQT  levels  is  TBD  in  the  follow  on  to  Project  A. 

D.  Examinees'  reactions:  motivation,  enjoyment,  boredom: 
Nothing  noteworthy. 

E.  Limitations  of  CAT-ASVAB  hardware  for  delivering  the 
test.  Additional  hardware  required:  None.  This  test  is 
up  and  running  on  the  HP.  NPRDC  has  two  forms  of  the 
executable  software. 

F.  Need  for  special  eqpt  with  special  maintenance? 

None 
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Outline  of  case  for  the  Map  Test  to  be  in  the  ECAT  Validation 

Description:  Map  is  a  somewhat  speeded  20  item,  multiple  choice 

test  from  the  spatial  battery  of  Project  A.  On  it, 
the  examinee  uses  a  schematic  map  and  explicit 
compass  directions  to  infer  implicit  ones.  Every 
item  has  the  same  eight  compass  directions  as  the 
response  choices.  Directions  and  a  sample  item 
are  given  below. 


MAP  TEST 


This  tGSt  medsures  how  well  you  cen  find  your  way  around  a  landscape,  using  a  map  and  your  knowledge 
of  compass  directions. 

There  are  two  parts  in  this  test.  For  each  part,  there  is  a  map  showing  an  aerial  view  of  the  ground  The 
map  shows  places  like  a  barracks,  a  campsite,  a  forest,  a  lake.  etc.  For  each  test  question,  you  will  be  given 
some  indication  as  to  which  direction  is  north,  west,  east  or  south.  For  example,  you  may  be  told.  'The 
barracks  are  due  SOUTH  of  the  storage  tanks  ”  Now  you  know  which  direction  is  South  and  you  can  use 
that  to  figure  out  all  the  other  directions.  Remember,  as  you  face  North,  East  is  always  to  your  right. 

Next,  you  will  be  told  that  you  need  to  get  from  one  place  to  another,  for  example,  from  the  forest  to  the 
campsite.  You  must  decide  which  direction  you  would  have  to  travel  to  get  there,  and  then  fill  inthe  circle  next 
to  the  correct  answer. 


Look  at  Example  1  below.  In  Example  1 .  you  are  told  that  the  shed  is  directly  North  of  the  tree  and  that  you  are 
at  the  storage  tank.  If  the  shed  is  North  of  the  tree,  then  which  direction  do  you  go  to  get  from  the  storage 
tank  to  the  tent?  You  must  travel  South  (S),  so  circle  ©  next  to  this  answer,  has  been  filled  in.  Please  study 
this  example  for  a  minute  to  see  for  yourself  why  South  is  the  correct  answer,  and  then  work  Example  2 
on  your  own. 


Example  Map: 


Ttflt 

TrM 

A 

H 

-X. 

$tor»o* 

ShM) 

Tsnk 

e 

1,  The  ihed  is  due  north  of  the  tree.  You  are  at  the  storage  tank.  Which  direction  must  you  travel  to  reach 
the  tent? 


©N  ©NE  ®E  ®SE  #5  ®SW  ©W  ®NW 


2.  The  tent  is  due  west  of  the  storage  tank.  You  are  at  the  storage  tank.  Which  direction  mutt  you  travel 
to  reach  the  tree? 

©N  ©NE  ®E  ®SE  ©S  ®SW  ©W  ®NW 

The  answer  for  Example  2  is  ©  .  for  NW  or  Northwest.  If  the  tent  is  west  of  the  storage  tank,  then  the  shed 
is  north  of  the  storage  tank,  and  the  tree  is  northwest  of  the  storage  tank. 

Are  there  any  questions  about  how  to  complete  this  test?  Your  score  will  be  the  total  number  of  correct 
answers  Work  as  quickly  and  as  accurately  as  you  can.  You  will  have  12  minutes  to  work  on  this  test. 
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1.  Theoretical  development  of  construct  and  measure (s) 

A.  Definition  of  construct  (process Fes 1 ) 

Domains : 

Spatial  ability  -  Orientation.  This  test  measures  the 
ability  to  maintain  one's  bearings  with  respect  to  the 
compass  despite  frequent  changes  in  orientation. 

Deductive  reasoning.  Given  an  explicit  direction  from 
one  site  on  the  map  to  another  in  the  item  stem,  the 
orientation  of  the  compass  can  be  deduced  and  used  to 
deduce  the  direction  from  any  third  site  to  any  fourth. 

How  is  this  construct  related  to  other  constructs/factors 
This  construct  was  not  included  in  Project  A's  expert 
judgment  task  (Wing,  Peterson,  &  Hoffman,  1984) ,  so  its 
location  in  the  hierarchical  map  of  the  predictor  space 
(Peterson,  1987,  p.  118)  did  not  emerge  from  that  work. 
See  below  Construct  Validity,  3. A. 

B.  Taxonomv/tvpes/cateaories  of  items 

Items  vary  in  the  extent  to  which  non-cardinal  directions 
are  involved  in  the  given  direction  and  the  question. 
Except  for  the  names  of  the  sites  (the  sites  are  clearly 
depicted  and  labelled  on  the  map)  and  the  specified 
directions,  the  syntax  of  the  item  stems  is  identical, 
with  one  minor  exception. 

How  the  test  is  scored/scaled;  Total  correct 

How  are  item  type  and  difficulty  jointly  sampled 
The  degree  of  difference  between  the  given  direction  and 
North  affects  item  difficulty,  as  well  as  the  degree  of 
difference  between  the  given  and  deduced  directions. 
Distribution  of  items  over  those  conditions  awaits  an 
examination  of  the  item  specifications. 

r.  fig>ATnina  differences  across  subgroups  in  experiences 
relevant  to  anv  type  of  item  or  in  offensiveness  of 
content  (e.a..  experience  with  computers) 

Offensiveness  is  not  an  issue,  as  the  items  are  almost 
content-f ree .  Experience  with  maps,  compass  directions, 
and  abstract  diagrams  may  favor  higher  SES  examinees. 
Rural  and  urban  examinees  may  have  experiences  that  could 
help,  but  those  experiences  are  likely  to  be  different. 

D.  Basis  for  expecting  incremental  validity  in  specific 

jobs;  The  Map  Test  resembles  the  common  soldiering  tasks 
(i.e.,  tasks  that  are  a  part  of  every  Army  enlisted  job) 
of  identifying  terrain  features  on  a  map  and  using  them 
to  navigate  over  land.  In  addition  such  tasks  are  a 
daily  part  of  the  job  in  Infantry,  Air  Defense,  Armor, 
Field  Artillery,  Air  Traffic  Control,  and  Intelligence, 
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where  it  is  essential  to  the  work  to  be  able  to  maintain 
directional  orientation  and  to  determine  locations  by 
using  landmarks  and  known  directions. 


2.  Precision  of  measurement 

A.  Test’s  consistency  across 

Different  forms;  Map  has  one  form  at  present 

Testing  occasions;  .78  in  499  incumbents 

levels  of  total  test  score  f score  conditional  SEM) ;  To 
be  determined  under  the  contract  following  up  Project  A 

B.  For  power  or  speeded  test;  internal  consistency 
Alpha  =  .89;  split  half  (Spearman-Brown)  =  .90 

c.  Variation  across  subgroups  in  score-conditional  SEM: 

To  be  computed  under  the  contract  that  follows  up 
Project  A. 

D.  If  score  is  model-based.  what*s  the  model-based 
precision?  N.  A. 

E.  Subgroup  differences  in  item  functioning; 

Fairness  analyses  will  be  carried  out  under  the  contract 
that  follows  up  Project  A.  Means  and  s.d.'s  of  the 
number  correct  measure  by  group  in  the  Concurrent 
Validation  were  as  follows: 


Group 

n 

X 

s.d. 

Whites 

6,007 

9.32 

5.53 

Blacks 

2,550 

4.16 

3.47 

Hispanics 

335 

6.32 

4.96 

Other 

355 

7.01 

5.40 

Females 

885 

6.32 

4.90 

Males 

8,362 

7.84 

5.56 

3.  Validity 

A.  Construct  validity:  correlations  with  other  tests  of 
similar  things.  Intercorr.  with  ASVAB; 

Spatial  and  general  abilities:  Page  81  shows  Map  loading 
almost  as  heavily  on  the  ASVAB  factor  as  on  the  Spatial 
one.  Its  highest  correlates  among  ASVAB  subtests  are 
MC,  AR,  MK,  and  GS  (p.  86) . 

B.  Unigueness  re  ASVAB.  How  assessed? 

The  squared  multiple  R  with  all  ASVAB  subtests  is  .44. 

In  terms  of  internal  consistency  (.90)  and  retest 
reliability  (.78),  uniqueness  is  .46  and  .34,  in 
order . 
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c.  Tncremental  validity  re  ASVAB?  IinproveTnent  in  SD  of  the 
predicted  criterion?  How  many  other  tests  were 
considered  when  this  one  was  validated?  In  analyses  of 
TOW  gunnery  data,  all  ASVAB  subtests,  and  all  Project  A 
computerized  and  spatial  tests.  Map  did  not  account  for 
unique  variance  nor  enter  significantly  into  a  stepwise 
regression  (p.  100) .  By  the  end  of  June,  we  expect  to 
have  data  evaluating  the  validity  of  Map  in  predicting 
performance  in  land  navigation  in  a  sample  of  300 
canditates  for  Special  Forces. 

n.  Subgroup  differences  in  incremental  validity  or  in  std 
err  of  prediction  of  the  criterion? 

To  be  calculated  from  the  Project  A  data  in  the  follow-on 
contract . 

F.  .Tnh  faTnilies  and  Patterns  of  validities  for  them. 

Similarity  of  iob  families  in  other  Services.  Expected 
utility  in  terms  of  numbers,  attrition,  training  costs, 
attractiveness  to  applicants:  The  combat  MOS  in  which 
map  reading  and  land  navigation  are  critical  tasks  are 
large  fill,  costly-to-fill  MOS.  Intelligence  MOS  are 
small,  costly  to  fill,  and  important.  For  incremental 
validity  of  a  composite  of  Project  A  spatial  tests 
against  major  outcomes  in  nine  MOS,  see  above  pp.6  and  7. 

4 .  Ecmatinq 

A.  Problems  in  developing  equivalent  forms  fe.q. .  How  large 
is  the  potential  item  pool?  Would  pretesting  and  vetting 
of  new  items  be  needed?  Could  item  devel  be  automated?) 
The  pool  of  potential  items  is  limitless:  by  varying  the 
map  alone,  the  item  stems  alone,  or  both,  we  could  create 
a  vast  pool  of  items.  Item  development  from  an  algorithm 
on  software  seems  practical.  Pretesting  of  items 
probably  would  be  needed  to  assure  that  item  statistics 
were  as  expected. 

B.  Types  of  samples  needed/acceptable  for  initial  ecmatinq? 
Testing  conditions  needed?  Status  as  applicant  or 
recruit  is  probably  not  as  important  as  the  range  of 
abilities  in  the  sample.  It  should  be  as  broad  as  that 
in  the  applicant  population  on  which  the  tests  will  be 
used. 

C.  Subgroup  dependence  of  eouatinast  To  be  determined  once 
equating  research  is  undertaken. 

D.  Practicality  of  pencil-and-oaper  versions:  The  present 
version  is  p/p  and  has  been  administered  to  60,000+ 
examinees  in  Project  A  without  difficulty. 
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5.  Feasibility  for  operational  S/C; 


A.  Min. /Max,  admin  times.  Tiroes  for  instrux/test  for  low 
and  high  AFOT  examinees:  Twelve  minutes  are  allowed  for 
working  the  items.  Time  for  instructions  is  about  5  min. 

B.  Susceptibility  to 

Cheating,  compromise;  Given  a  large  item  pool,  these 
should  be  low. 

Irrelevant  response  strategies;  Some  examinees  use  a 
strategy  of  physically  rotating  the  map.  Whether  that 
helps  is  not  known.  The  strategy  of  using  the  given 
direction  to  determine  North,  then  holding  a  finger  or 
thumb  pointed  toward  that  North,  seemed  to  help  one 
ARI  researcher  score  2  s.d.  above  the  mean  on  Map  for 
first  tour  soldiers,  but  that  strategy  still  requires 
the  examinee  to  know  what  to  do  with  the  knowledge  of 
where  North  is. 

Coaching;  Unknown 

Practice:  On  retesting,  502  incumbents  increased  their 

mean  score  by  .08  s.d.  (p.  97). 

C.  Floor /ceiling  effects.  Performance  of  lov/high  AFQTs; 
With  8  response  choices  and  20  items,  chance  perform¬ 
ance  is  2.5  correct.  That  is  less  than  one  s.d.  below 
the  mean  of  7.67  (s.d.  5.51,  n  >9,300).  Separate 
analyses  have  not  been  done  yet  with  AFQT  levels. 

D.  Examinees*  reactions;  motivation,  emovment, _ boredom^ 

Nothing  exceptional 

E.  Limitations  of  CAT-ASVAB  hardware  for  delivering  the 
test.  Additional  hardware  required; 

This  test  is  up  on  the  HP.  The  graphics  are  very  clear, 
but  the  map  is  a  bit  crowded.  No  other  hardware  is 
required.  The  regular  number  or  letter  keys  may 
suffice  to  enter  the  eight  response  choices. 

F.  Need  for  special  egpt  with  special  maintenance? 

None. 
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Draft  of  the  case  for  using  the  Nazes  test  in  the  ECAT  Validation 


Description:  Mazes  is  a  somewhat  speeded  24-item  test  from  the 

spatial  battery  of  Project  A  where  the  examinee 
selects  the  one  entrance  to  a  rectangular  maze  that 
leads  to  an  exit.  For  directions  and  a  sample 
item ,  see  below • 


MAZE  TEST 


Thi$  tt«  mttiurtt  how  wtll  you  can  find  your  way  through ,a  maxa.  Each  maxa  In  tha  taat  hai  four  amrancaa* 
labaiad  A.  t.  C,  and  D.Tha  axit*  ara  markad  with  an  arrow  pointing  out  and  a  atar  Only  ona  of  tha  four 
•ntraneat  laads  to  an  axit  out  of  tha  maxa.  You  must  figura  out  which  tntranca  (A,  C«  or  D)  laadi  to  a 
way  out  Mark  your  anawar  by  filling  in  tha  correct  lattarad  circia  balow  tha  maxa. 


Look  at  tha  axampla  balow.  Which  ontranct  laada  to  a  way  out  of  tha  maxa?  Fill  in  tha  corract  lattarad 
circia  balow  tha  maxa. 


EXAMPLE: 


•  • 


0  0©© 


Tha  corract  anav^^r  to  thia  axampla  ia  B.  B  ia  tha  only  antranea  that  laada  to  a  way  out  of  tha  Mxa.-Tha  dotted 
line  in  the  picture  below  ahowa  how  tha  entrance  markad  B  ia  tha  only  ona  that  laada  to  an  exit  from  tha  maxa 
To  mark  your  anawar.  you  ahoufd  have  filled  in  the  circle  B  located  below  tha  maxa,  at  ahown.  Plaaaa  do 
not  make  any  other  partcil  marka  on  tha  taat. 


•  m 


GO®® 


Ara  thara  any  quettient  about  hoa»  ia  eomplata  this  taat?  Your  aeora  will  ba  tha  total  nombar  of  corract 
antwtfi.  Work  at  quickly  and  at  aeeurattly  at  you  can.  You  will  havt  8Vi  minuttt  to  work  on  thit  tatt. 


DO  NOT  BEGIN  UNTIITOIO  TO  00  SO. 
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1.  Theoretical  development  of  construct  and  measure (s) 

A.  Definition  of  construct  (process r es 1 ) 

Domain;  Primarily  spatial  ability 

Spatial  Sub-domain;  It  is  not  clear  which  of 
Alderton’s  big  four  spatial  factors  best  fit(s)  the 
abilities  measured  by  this  test.  Perceptual  Speed 
is  the  closest,  Mazes  involving  speeded  visual 
search.  But  the  stimuli  are  complex  in  some  ways, 
and  the  abilities  to  scan  a  field,  stay  on  a  path,  and 
to  remember  which  paths  have  been  tried  already  (or  to 
distinguish  signal  from  noise  and  suppress  the  noise)  are 
involved,  too.  TASP,  at  its  April  meeting,  put  Mazes 
under  the  non-committal  heading  "Spatial  Relations." 

PDRI,  which  developed  the  Project  A  predictor  battery, 
put  Mazes  under  Spatial  Visualization,  which  TASP 
explicitly  did  not. 

How  is  this  construct  related  to  other  constructs/factors 
See  below  3. A.  Construct  Validity 

B.  Taxonomy /types /categories  of  items 

Item  difficulty  varies  with  lengths  of  the  paths,  in 
terms  of  number  of  turns  and  number  of  blind  alleys. 

How  the  test  is  scored/scaled 

One  score  is  used;  total  number  correct. 

How  are  item  type  and  difficulty  jointly  sampled 

To  be  determined  in  the  upcoming  follow  on  to  Project  A. 

C.  Seeming  differences  across  subgroups  in  experiences 
relevant  to  anv  type  of  item  or  in  offensiveness  of 
content  (e.a. .  experience  with  computers) 

Being  content-free,  the  items  are  not  offensive.  There 
may  be  SES-related  differences  in  exposure  to  posers, 
puzzles,  or  maze-like  video  games. 

D.  Basis  for  expecting  incremental  validity  in  specific 
jobs;  ASVAB  does  not  currently  measure  the  spatial 
abilities  which  rational  analysis  suggests  are  needed  in 
many  Army  jobs.  Project  A  experts  (Wing,  Peterson,  and 
Hoffman,  1984)  judged  that  measures  of  the  visualization 
and  scanning  constructs  would  have  good  validities 
against  a  variety  of  inspecting,  troubleshooting,  and 
repairing  tasks  (median  .31),  as  well  as  using  maps 

and  controlling  air  traffic  (*.38) 
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2.  Precision  of  measurement 


A.  Test»s  consistency  across 

Different  forms;  Mazes  has  one  form  at  present 

Testing  occasions;  Retest  reliability  =  .70  in  499 
incumbents 

Levels  of  total  test  score  Tscore  conditional  SEM) : 

To  be  determined  under  the  contract  that  follows  up 
Project  A. 

B.  For  power  or  speeded  test,  internal  consistency; 

Odd-even  reliability:  .96  (N  *  9,344  incumbents); 

Alpha:  .89  (N  =  >  9,331  incumbents). 

C.  Variation  across  subgroups  in  score-conditional  SEM: 

To  be  determined  under  the  contract  that  follows  up 
Project  A. 

D.  If  score  is  model-based,  what's  the  model-based  precision 

N.  A. 

E.  Subgroup  differences  in  item  functioning 

Fairness  analyses  will  be  carried  out  under  the  contract 
that  follows  up  Project  A.  Means  and  s.d. 's  of  the 
number  correct  by  group  in  the  Concurrent  Validation 
were  as  follows: 


Group 

n 

X 

S.d. 

Whites 

6,007 

17.64 

4.31 

Blacks 

2,551 

13.36 

4.50 

Hispanics 

335 

16.92 

4.03 

Other 

354 

16.97 

4.81 

Females 

885 

15.04 

4.81 

Males 

8,362 

16.55 

4.74 

3.  Validity 

A.  Construct  validity:  correlations  with  other  tests  of 
similar  things.  Intercorr.  with  ASVAB 

Spatial  factor/construct 

On  page  81,  the  six  spatial  tests  in  the  Concurrent 
Validation  formed  the  second  factor  in  a  principal 
factor  analysis  of  ASVAB  subtests  and  new  Project  A 
tests . 

Relation  of  Mazes  to  G 

Page  81  shows  Mazes  to  have  the  lowest  loading  of 
the  spatial  tests  on  the  factor  consisting  of  the 
eight  unspeeded  tests  of  ASVAB;  viz.,  .12. 
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Spatial  speed: 

Correlated  with  ASVAB  subtests.  Mazes'  highest 
correlations  are  with  MC,  MK,  and  AR,  while  the 
lowest  correlations  are  with  WK  and  the  two 
speeded  subtests,  PC  and  NO  (p.  86).  Factor 
analyses  of  the  six  spatial  tests  from  Project  A 
gave  a  two-factor  solution  (p.  88)  where  Mazes 
clusters  with  the  more  speeded  spatial  tests, 
while  a  three-factor  solution  (p.  89)  groups  it 
with  Object  Rotation,  the  simplest,  most  speeded  of 
the  tests. 

other;  On  p.  89,  Mazes  is  seen  to  have  a  strong 
secondary  loading  on  a  factor  with  Assembling 
Objects  and  Reasoning. 

Psvchomotor  abilities 

In  factor  analyses  involving  spatial  and 
computerized  tests  from  Project  A  (p.  81  including 
ASVAB  and  p.  83  without  ASVAB) ,  Mazes  has  the 
highest  loading  of  the  spatial  tests  on  the 
psychomotor  factor.  In  a  separate  sample  of  95 
Armor  officers  who  took  the  spatial  and  psychomotor 
tests  (p.  84),  Mazes  loaded  more  strongly  with 
tracking  than  with  the  other  five  seemingly 
spatial  tests.  These  latter  two  results 
are  consistent  with  the  observation  that  many 
examinees  physically  trace  paths  through  the 
mazes  with  their  fingers. 

B.  Unicfueness  re  ASVAB.  How  assessed? 

The  squared  multiple  R  of  all  ASVAB  subtests,  given 
before  enlistment,  with  Mazes  was  .22  for  7,329 
first  tour  soldiers.  Uniqueness  was  estimated  to  be 
.74  and  .48  for  internal  consistency  and  retest, 
respectively . 

C.  Incremental  validity  re  ASVAB?  Improvement  in  SD  of  the 
predicted  criterion?  How  many  other  tests  were 
considered  when  this  one  was  validated?  Incremental 
validities  of  all  computerized  and  spatial  predictors 
from  Project  A  were  examined  in  GLM  analyses  by  Silva 
(1989,  unpublished).  The  sample  was  311  trainees  who 
had  taken  ASVAB  as  applicants,  the  Project  A  tests 
during  their  first  three  days  of  Service  (1986/87),  and 
then  performance  tests  on  their  anti-tank  gunnery  skills 
about  twelve  weeks  later.  The  criterion  measure  used 
here  was  accuracy  on  the  fifth  event  (event  =  set  of  10 
trials) ,  which  was  the  first  event  on  which  a  gunner 
could  attain  qualification.  Correlations  of  all  Project 
A  tests,  AFQT,  and  ASVAB' s  Combat  Composite  with  the 
criterion  were  first  examined.  Variance  in  the 
criterion  accounted  for  by  two  sets  of  tests  -  all 
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Project  A  and  all  ASVAB  -  was  computed  separately  and 
then  same  analysis  was  run  on  the  Project  A  tests 
with  the  criterion  variance  associated  with  ASVAB 
removed.  Finally,  the  Project  A  tests  were  entered  into 
a  stepwise  linear  regression  on  the  residual  (unrelated 
to  ASVAB)  variance . 

Mazes  correlated  .108  with  the  criterion,  compared  with 
.179  for  the  Combat  Composite  (CS  +  AR  +  MC  +  AS)  and 
.145  with  AFQT.  In  various  GLM  analyses  with  all  other 
Project  A  tests  entered  (p.  100) ,  Mazes  did  not  account 
for  significant  unique  variance  nor  significantly  enter 
into  the  stepwise  regression. 

In  1988,  a  battery  of  four  Project  A  tests  —  One-  and 
Two-hand  Tracking,  Mazes,  and  Orientation  —  was 
installed  at  two  training  bases  for  a  pilot  test  on 
students  in  the  IIH  anti-tank  and  19K  tank  crew  courses. 
In  samples  of  these  students  (pp.  101-2) ,  the  validity  of 
an  optimally  weighted  combination  of  all  ASVAB  subtests 
against  scores  on  TOW  and  anti-tank  gunnery  simulators 
was  .266  and  .480,  respectively.  Mazes  had  a  validity  of 
.020  (n.s.)  and  .308.  In  both  of  the  1988  samples. 

Mazes  did  account  for  significant  unique  variance  and 
enter  significantly  into  stepwise  regressions  on  the 
residual ized  performance  variance.  In  these  regressions, 
it  added  .017  and  .024  in  variance  accounted  for. 

D.  Subgroup  differences  in  incremental  validity  or  in  std 
err  of  prediction  of  the  criterion? 

For  the  sample  in  2. A  above,  there  were  88  blacks  and 
444  whites.  Score-conditional  SEP  are  as  follows; 

SD  units  of  #  correct 

on  Mazes;  -2  -1  0  +1  +2 

SEP  W  (n  =  444)  1.5  0.9  0.6  0.8  1.4 

SEP  B  (n  =  88)  3.3  2.2  2.2  3.3  4.9 


E.  Job  families  and  patterns  of  validities  for  them 

For  incremental  validities  of  a  composite  of  Project  A 
spatial  tests  against  major  outcomes  in  nine  MOS,  see 
above  pp.  6-7. 

The  value  of  new  predictors  for  classification  will  be 
more  obvious  at  the  level  of  relevant  tasks,  not  at  the 
level  of  whole  jobs. 

similarity  of  -iob  families  in  other  services 
Most  Army  jobs  have  counterparts  in  the  Marines.  Tasks 
that  involve  inspecting,  troubleshooting,  and  repairing 
electronics  and  mechanical  systems  have  counterparts  in 
all  other  Services. 
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Expected  utility  in  terms  of  numbers,  attrition, 
training  costs,  attractiveness  to  applicants 
Army's  combat  MOS  (Infantry,  Armor,  Field  Artillery, 
Combat  Engineer)  require  large  numbers  and  are  the  most 
critical  (on  a  scale  that  equates  criticality  with 
closeness  to  combat),  but  are  hard  to  fill.  For  at  least 
one  MOS  in  Infantry,  Armor,  and  Artillery  (IIH,  19K,  and 
13F) ,  success  in  "gunnery"  is  objectively  measured  near 
the  end  of  entry-level  training.  For  one  other  Infantry 
MOS,  lie  Mortar  Crewman,  firing  is  supposed  to  be  tested 
objectively  twice  a  year  in  the  active  forces. 
Requalification  on  weapons  systems  is  carried  out 
periodically,  either  in  live-fire  or  on  simulators, 
in  the  active  forces. 

4 .  Equating 

A.  Problems  in  developing  equivalent  forms  (e.q..  How  large 
is  the  potential  item  pool?  Would  pretesting  and  vetting 
of  new  items  be  needed?  Could  item  devel  be  automated?) 
Equivalent  forms  should  be  easy  to  create.  First,  "new" 
items  could  be  made  by  rotating  the  present  items  90, 

180,  or  270  degrees.  Then,  an  algorithm  for  generating 
items  could  be  written  that  varies  the  number  of  turns, 
distance  between  turns,  etc.  New  items  should  have  some 
pilot  testing  to  make  sure  that  their  statistics  are  as 
expected . 

B.  Types  of  samples  needed/acceptable  for  initial  equating? 
Testing  conditions  needed? 

Filling  cells  for  minorities,  women,  and  low  AFQTs  will 
influence  sample  size.  Conditions  suitable  for  any 
pencil/paper  or  computerized  testing  should  suffice. 

C.  Subgroup  dependence  of  equatings 

To  be  determined  when  equating  research  is  undertaken. 

D.  Practicality  of  pencil-and-paper  versions 

High.  Mazes  originated  as  a  pencil-and-paper  test. 

5.  Feasibility  for  operational  S/C 

A.  Min. /Max,  admin  times.  Times  for  instrux/test  for  low 
and  high  AFOT  examinees 

Mazes  is  a  speeded  test  with  5.5  minutes  allowed  to 
work  the  scored  items.  Time  for  directions  is  about 
5  minutes. 

B.  Susceptibility  to: 

Practice;  In  502  incumbents,  mean  scores  on  Mazes  rose 
.20  s.d.,  from  16.54  to  17.46. 
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Irrelevant  response  strateaies/coachinq; 

One  tactic  that  works  for  some  examinees,  viz.,  work  Back 
from  the  exits  instead  of  forward  from  the  entrances, 
seems  not  to  change  the  ability  being  measured.  For 
items  where  more  than  one  exit  can  be  reached  by  the 
correct  entrance,  that  strategy  could  give  an  advantage. 

Cheat ina/compromise ;  None  known 

C.  Floor /ceiling  effects?  Performance  of  low/hiah  AFQTs 
on  the  test?  Examinees  averaged  1.6  s.d.  below  the 
ceiling  in  the  Concurrent  Validation  (n  >  9,331  incum¬ 
bents)  .  In  research  on  tank  and  anti-tank  gunnery  (pp. 
100  -  103) ,  scores  on  Mazes  had  enough  variance  to 
correlate  significantly  with  gunnery  performance. 

D.  Examinees*  reactions;  motivation,  enjoyment,  boredom: 
Nothing  remarkable 

F.  Limitations  of  CAT-ASVAB  hardware  for  delivering  the 

test.  Additional  hardware  recmired;  Uses  a  great  deal 
of  space.  Needs  no  additional  hardware 

F.  Need  for  special  eopt  with  special  maintenance;  None. 
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Outline  of  case  for  Orientation  to  be  in  the  ECAT  Validation 

Description?  Orientation  is  a  somewhat  speeded  2 4 “item  test 

from  the  Project  A  spatial  battery  where  examinees 
select  an  answer  depicting  what  a  figure  will  look 
like  after  it  is  rotated  in  the  plane  of  the  page. 
Directions  and  sample  items  are  given  below  and  on 
the  next  page. 


ORIENTATION  TEST 

This  is  a  test  of  your  ability  to  imagine  how  an  object  will  look  when  it  has  been 
turned  or  rotated. 


Please  look  at  the  sample  problem  below,  h  contains  a  picture  with  a  frame  around 
it.  The  bottom  of  the  frame  has  a  circle  with  a  dot  inside,  carved  into  the  frame. 


If  you  examine  the  frame  and  the  picture  inside  it,  you  will  notice  that  the  bottom 
of  the  frame,  marked  by  the  circle  with  a  dot,  does  not  match  up  with  the  bottom 
of  the  picture.  Imagine  that  only  the  frame  can  be  turned,  and  the  picture  inside  can 
not  be  moved.  Then,  to  match  up  the  bottom  of  the  frame  with  the  bottom  of  the 
picture,  you  would  turn  the  frame  until  the  circle  with  a  dot  is  located  at  the 
bottom  of  the  picture. 

The  circle  with  the  dot  inside  moves  with  the  frame  as  you  turn  it,  so  after  the 
frame  has  been  turned  and  you  stand  back,  facing  it,  the  dot  will  appear  to  be  In 
a  different  position  in  the  circle. 

Your  task  is  to  figure  out  exactly  what  the  circle  with  the  dot  inside  will  look 
like  in  its  new  position,  after  the  frame  has  been  turned. 

To  understand  how  this  works,  look  at  the  solution  to  Example  1  on  the 
next  page. 
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EXAMPLE  1: 


1 


©  Q  ®  Q 

®  o  ®  o 

•  o 


Remember,  the  first  step  is  to  mentally  turn  the  frame  so  that  the  circle  with 
the  dot  inside  is  located  at  the  bottom  of  the  picture.  The  arrow  shows  that,  for 
Example  1.  the  frame  should  be  turned  to  the  left  one^arter  of  a  full  turn. 


As  the  frame  is  turned,  it  carries  with  it  the  circle  with  the  dot  inside.  So,  in  Example  1. 
after  the  frame  has  been  turned,  the  circle  with  the  dot  inside  will  look  like  this:  ^ 
with  the  dot  at  the  bottom  of  the  circle.  TWs  is  the  answer  shown  in  answer  W 
choice  C,  so  the  C  has  been  darkened. 


Thus,  to  solve  each  problem,  you  need  to: 

1.  Mentally  turn  or  rotate  the  frame  until  the  bottom  of  the  frante  (the  circle  wrth 
the  dot)  is  located  at  the  bottom  of  the  picture,  and  then 

2.  Figure  out  what  the  circle  with  the  dot  inside  will  look  like  in  its  new  position. 


Now  try  to  work  Example  2  on  your  own. 


For  Example  2,  you  would  turn  the  frame  one^arter  turn  to  the  right,  so  that  the 
circle  with  the  dot  inside  is  located  at  the  bottom  of  the  picture.  After  you  have  done 
tNs,  the  circle  with  the  dot  inside  will  look  like  this:  ^  in  its  new  position. 

Answer  B  shows  the  circle  with  the  dot  mside  in  this  position,  so  you  should 


have  darkened  B. 


Are  there  any  questions  about  how  to  complete  tNs  test?  Your  score  will  be  the  total 
number  of  correct  answers.  Work  as  quickly  and  accurately  as  you  can.  You  will 
have  10  rranutes  to  work  on  tNs  test 


DO  NOT  BEGIN  UNTIL  TOLD  TO  DO  SO. 
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1.  Theoretical  development  of  construct  and  measure fs) 

A.  Definition  of  construct  (process Tesl) 

Domain:  Spatial  abilities 

Spatial  Sub-domain:  Spatial  Relations 

Orientation  is  the  ability  to  keep  one's  bearings  with 
respect  to  the  environment  despite  changes  in  perspective 
or  relative  location.  The  items  in  this  test  also 
require  the  ability  to  identify  pictured  objects.  That 
part  of  the  task  is  intended  to  be  very  easy. 

How  is  this  construct  related  to  other  constructs/factors 
For  factor  analytic  evidence,  see  below  3. A. 

R.  Taxonomv/tvpes/cateaories  of  items 

Difficulty  was  varied  by  manipulating  the  amount  of 
rotation  required  for  correct  solution. 

How  the  test  is  scored/scaled?  Total  correct. 

How  are  item  tvoe  and  difficulty  jointly  sampled 
To  be  determined  by  examination  of  the  items'  design 
and  statistical  parameters. 

C.  Seeming  differences  across  subgroups  in  experiences 
relevant  to  anv  tvoe  of  item  or  in  offensiveness  of 
content  (&.a. .  experience  with  computers):  None  apparent 

D.  Basis  for  expecting  incremental  validity  in  specific  jobs 
ASVAB  does  not  now  measure  spatial  abilities.  Field 
observations  by  Project  A  researchers  suggested  that 
maintaining  one's  spatial  orientation  is  critical  in 
Army  jobs  such  as  Tank  Crew  and  Air  Defense  Artillery 
Crew  and  cn  tasks  requiring  use  of  maps  and  landmarks 
(i.e.,  in  many  jobs). 

2.  Precision  of  measurement 

A.  Test's  consistency  across 

Different  forms:  Orientation  has  one  form  at  present 

Testing  occasions;  .70  in  499  incumbents 

Levels  of  total  test  score  (score  conditional  SEM) ; 

These  analyses  have  not  yet  been  carried  out  on  data 
from  Project  A.  The  primary  analyses  of  interest 
will  be  against  criteria  of  map  reading,  target 
identification,  navigation,  and  general  hands  on 
proficiency. 

B.  For  power  or  speeded  test;  internal  consistency 

In  >  9,300  incumbents,  Alpha  =  .89;  split-half  (Spearman- 
Brown)  =  .89 
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c. 


Variation  across  subgroups  in  score-conditional  SEM 
To  be  determined  in  the  contract  that  follows  up 
Project  A. 

n.  If  score  is  model-based.  what*s  the  model-based  precision 

N.A. 


E.  Subgroup  differences  in  item  functioning 

Fairness  analyses  will  be  carried  out  under  the  contract 
that  follows  up  Project  A.  Means  and  s.d.'s  of  the 
number  correct  measure  by  group  in  the  Concurrent 
Validation  were  as  follows: 


Group 

n 

X 

s.d. 

Whites 

6,003 

12.52 

6.28 

Blacks 

2,553 

7.79 

4.46 

Hispanics 

334 

9.72 

5.73 

Other 

355 

10.45 

6.14 

Females 

887 

9.75 

5.35 

Males 

8,358 

11.17 

6.24 

3.  Validity 

a.  ronstrur-t  validity:  correlations  with  other  tests  of 
similar  things.  Intercorr.  with  ASVAB 

Snatial  factor/construct :  On  p.  81,  the  six  spatial 
tests  in  the  CV  formed  the  second  factor  in  a 
pj^irjcipal  factor  analysis  of  ASVAB  subtests  and  new 
Project  A  tests.  Orientation  loaded  .5  on  the 
spatial  factor  and  .34  on  the  factor  formed  by 
the  unspeeded  tests  of  ASVAB.  The  ASVAB  subtests 
which  it  correlates  with  most  highly  (>  .40)  are 
MC,  MK,  and  AR.  The  squared  multiple  R  with  all 
ASVAB  subtests  is  .29. 

B.  Uniqueness  re  ASVAB.  How  assessed? 

Orientation  had  a  uniqueness  of  .60  and  .41  relative 
to  internal  consistency  and  retest  reliability. 

r,  Tnnremental  validity  re  ASVAB?  Improvement  in  sp  of  the 
predicted  criterion?  How  many  other  tests  were 
considered  when  this  one  was  validated? ^  Incremental 
validities  of  all  computerized  and  spatial  predictors 
from  Project  A  were  examined  in  GLM  analyses  by  Silva 
(1989,  unpublished).  The  sample  was  311  trainees  who 
had  taken  ASVAB  as  applicants,  the  Project  A  tests 
during  their  first  three  days  of  Se^ice  (1986/87),  and 
then  performance  tests  on  their  anti-tank  gunnery  skills 
about  twelve  weeks  later.  The  criterion  measure  used 
here  was  accuracy  on  the  fifth  event  (event  —  set  of  10 
trials),  which  was  the  first  event  on  which  a  gunner 
could  attain  qualification.  Correlations  of  all  Project 
A  tests,  AFQT,  and  ASVAB 's  Combat  Composite  with  the 


29 


criterion  were  first  examined.  Variance  in  the 
criterion  accounted  for  by  two  sets  of  tests  -  all 
Project  A  and  all  ASVAB  -  was  computed  separately  and 
then  same  analysis  was  irun  on  the  Project  A  tests 
with  the  criterion  variance  associated  with  ASVAB 
removed.  Finally,  the  Project  A  tests  were  entered  into 
a  stepwise  linear  regression  on  the  residual  (unrelated 
to  ASVAB)  variance. 

Orientation  correlated  .242  with  the  criterion, 
compared  with  .179  for  the  Combat  Composite  (CS  +  AR  + 

MC  +  S)  and  .145  with  AFQT.  In  various  GLM  analyses 
with  all  ASVAB  and/or  other  Project  A  tests  entered  (p. 
100) ,  Orientation  did  contribute  significant  unique 
variance  (p  =.015  for  variance  not  overlapping  with 
Project  A  tests  nor  ASVAB  subtests) .  It  was  the  last 
predictor  to  enter  significantly  in  the  stepwise 
regression  with  all  other  Project  A  tests  adding  .0117  to 
the  r2  (p  =  .0453)  . 

In  1988,  a  battery  of  four  Project  A  tests  —  One-  and 
Two-hand  Tracking,  Mazes,  and  Orientation  —  was 
installed  in  two  training  bases  for  a  pilot  test  on 
students  in  the  IIH  anti-tank  and  19K  tank  crew  courses. 
In  samples  of  these  students  (pp.  101-2),  the  validity  of 
an  optimally  weighted  combination  of  all  ASVAB  subtests 
against  scores  on  TOW  and  anti-tank  gunnery  simulators 
was  .266  and  .480,  respectively.  Orientation  correlated 
.162  with  scores  on  the  TOW  simulator  and  .328  with 
speed/accuracy  on  the  simulator  of  tank  gunnery.  In  the 
1988  samples.  Orientation  did  not  account  for  unique 
variance  nor  enter  significantly  into  the  stepwise 
regressions.  By  itself,  it  accounted  for  less  than  1%  of 
the  residualized  variance,  and  combining  it  with  the 
other  three  predictors  variously  had  little  effect  on  R^. 

D.  Subgroup  differences  in  incremental  validity  or  in  std 
err  of  prediction  of  the  criterion? 

In  88  black  and  444  white  trainees  in  MOS  IIH,  TOW 
Gunner,  score  conditional  SEP  of  accuracy  in  the  first 
qualifying  table  (10  firings)  in  1988  were: 


Black 

White 

P 

+2  sd 

6.3 

1.3 

.862 

+1 

4.2 

0.8 

.198 

mean 

2.4 

0.6 

.000 

-1 

2.1 

0.9 

.000 

-2 

3.6 

1.4 

.000 

E.  Job  families  and  patterns  of  validities  for  them. 
Similarity  of  iob  families  in  the  other  Services. 

Expected  utility  in  terms  of  numbers,  attrition,  training 
costs,  attractiveness  to  applicants;  See  pp.  6-7. 
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Army's  combat  MOS  (Infantry,  Armor,  Field 

Artillery,  Combat  Engineer)  require  large  numbers  and  are 
the  most  critical  (on  a  scale  that  equates  criticality 
with  closeness  to  combat),  but  are  hard  to  fill. 

Attrition  from  Army  training  for  failures  of  ability 
is  rare.  Instead,  attrition  occurs  for  low  motiva¬ 
tion,  adjustment,  fitness,  discipline,  and  the  like. 

4 .  Equating 

A.  Problems  in  developing  equivalent  forms  (e.g..  How  large 
is  the  potential  item  pool?  Would  pretesting  and  vetting 
of  new  items  be  needed?  Could  item  devel  be  automated?) 
The  pool  of  potential  orientation  items  is  unlimited. 

By  varying  the  orientation  of  the  picture,  the  location 
of  the  circle  with  the  dot,  and  the  orientation  within 
the  circle  of  the  dot,  all  of  which  can  vary 
continuously,  we  could  create  a  very  large  pool. 

Analyses  of  existing  data  in  the  Project  A  database 
may  reveal  whether  the  relation  of  item  difficulty  to 
those  variables  is  tight  enough  to  make  pretesting  of 
new  items  unnecessary,  except  for  small  debugging  trials. 

B.  Types  of  samples  needed/acceotable  for  initial  eouatino? 
Testing  conditions  needed? 

Instructions  for  Orientation  make  use  of  an  enlarged 
sample  test  item,  which  has  two  pieces:  the  basic  item 
and  a  separate,  movable  frame  for  the  picture.  When 
this  test  is  given  to  large  groups  on  paper,  either 
more  than  one  enlarged  sample  item  is  needed,  or  else 
the  TA  needs  to  demonstrate  it  from  several  locations 
in  the  room,  so  everyone  can  see  it  clearly. 

C.  Subgroup  dependence  of  equating:  To  be  determined  when 
we  get  into  equating . 


D.  Practicality  of  oencil-and-paper  versions 
The  existing  version  is  on  paper. 

5.  Feasibility  for  operational  S/C 

A.  Min. /Max,  admin  times.  Times  for  instrux/test  for  low 
and  high  AFOT  examinees:  Ten  minutes  are  allowed  for 
doing  the  items.  The  present  instructions  for  examinees 
are  needlessly  long  and  unclear.  Administration  on 
computer  may  permit  the  verbal  instructions  to  be 
simplified  by  enabling  presentation  of  more  sample 
test  items.  Est.  time  for  present  instructions:  10  min. 
Because  the  test  has  been  administered  lock  step  to 
groups,  no  opportunity  has  been  given  for  times  to  vary. 
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B.  Susceptibility  to 


Cheating,  compromise:  Not  certain.  These  dangers  could 
be  reduced  by  developing  and  using  a  very  large  item  pool 

Irrelevant  response  strategies:  Individual  examinees 
believe  that  they  discover  helpful  strategies  (e.g., 
physically  rotate  the  page  to  see  what  the  stimulus 
looks  like  after  rotation.  That  changes  the  task  from 
mental  to  physical  rotation.).  Whether  any  such 
strategies  are  successful  remains  to  be  seen. 

Practice;  In  500  incumbents,  the  gain  was  .27  s.d. 

Coaching ;  Unknown . 

C.  Floor /ceiling  effects?  Performance  of  low/hioh  AFQTs 

on  the  test?  The  mean  of  11.02  and  s.d.  of  6.18  on  over 
9,000  cases  in  the  concurrent  validation  put  the  mean 
more  than  two  s.d.  below  the  maximum  possible  score  and 
.8  s.d.  above  chance. 

D.  Examinees*  reactions:  motivation,  enjoyment,  boredom: 
Nothing  noteworthy 


E.  Limitations  of  CAT-ASVAB  hardware  for  delivering  the 
test.  Additional  hardware  required;  No  additional 
hardware  would  be  required,  but  the  graphics  in  the 
present  test  are  likely  to  be  unclear  on  a  computer 
screen.  They  are  already  unclear  on  paper.  That 
fact,  and  the  likelihood  that  the  ability  tested 
does  not  depend  much  on  the  exact  nature  of  the 
picture  within  the  frame  of  each  item  suggest  that 
the  test  could  be  improved  by  simplifying  the  picture. 
The  ’’picture"  could  be  an  arrow,  and  the  task  then 
could  be  to  rotate  the  frame  until  the  circle  with 
the  dot  is  at  the  point  of  the  arrow. 

F.  Need  for  special  eaot  with  special  maintenance?  None 
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Outline  of  the  case  for  Reasoning  to  be  in  ECAT  Validation 


Description:  Reasoning  is  a  somewhat  speeded  30-item  test  from 

the  spatial  battery  of  Project  A.  Given  a  row  of 
four  figures  that  form  a  logical  series,  examinee 
selects  from  multiple  choices  the  one  that  is  the 
next  step  in  the  series.  Directions  and  a  sample 
item  are  given  below. 


REASONING  TEST 


This  test  has  rowKS  of  figures  like  these  shown  below.  Each  row  across  the  page  is  one  problem  for  you 
to  solve. 

Note  that  each  row  has  four  figures  on  the  left  called  FIGURE  SERIES  end  five  figures  on  the  right 
called  POSSIBLE  ANSWERS.  You  need  to  discover  what  the  pattern  is  in  the  FIGURE  SERIES  (what 
is  going  on  as  you  move  from  one  figure  to  the  next),  end  then  decide  which  one  of  the  POSSIBLE 
ANSWERS  would  appear  next  in  the  series.  Please  look  at  Example  1. 


FIGURE  SERIES  _ POSSIBLE  ANSWERS 


Example  1 

— 

— 

— 

— 

© 

© 

• 

© 

In  Example  1,  note  that  the  lines  in  the  FIGURE  SERIES  are  moving  lower  and  lower  in  the  boxes. 
The  first  line  is  almost  at  the  top,  and  as  you  go  from  square  to  square,  the  line  sinks  lower  and  lower. 
What  would  the  next  square  in  the  series  look  like?  Where  would  the  line  be?  The  line  would  be  near 
the  bottom  of  the  square,  as  in  answer  Choice  D.  So  the  correct  answer  is  0.  and  circle  0  has  been 
filled  in.  Now  complete  Example  2  on  your  own. 

FIGURE  SERIES 

POSSIBLE  ANSWERS 

■ 

■ 

■ 

■ 

Example  2 

■ 

■ 

■ 

■ 

■ 

0 

© 

© 

© 

© 

In  Example  2.  the  black  square  in  the  FIGURE  SERIES  is  moving  around  counterclockwise  inside 
the  larger  square:  it  starts  in  the  upper  left  corner,  goes  to  the  lower  left  corner,  to  the  lower  right 
corner,  and  to  the  upper  right  corner.  Its  next  move  will  put  it  back  in  the  upper  left  corner.  So  A  is  the 
correct  answer  and  you  should  have  filled  in  circle  A. 

For  every  problem,  you  must  first  look  at  the  FIGURE  SERIES  to  decide  what  the  pattern  is.  and  then 
choose  from  among  the  POSSIBLE  ANSWERS  the  one  that  comes  next  in  the  series. 

Are  there  any  questions  about  how  to  complete  this  test?  Your  score  will  be  the  total  number  of  correct 
answers.  Work  as  quickly  and  as  accurately  as  you  can.  You  will  have  12  minutes  to  work  on  this  test. 
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1.  Theoretical  development  of  construct  and  measure (s) 

A.  Definition  of  construct  (process [es] ) 

Domains: 

General  intelligence 

Inductive  reasoning:  This  test  measures  the 
ability  to  generate  and  test  hypotheses  about 
the  relationships  in  a  series  of  abstract  figures. 

Working  memory:  Solving  a  problem  requires 
keeping  the  relationships  among  successive 
figures  in  memory  in  order  to  find  the  principle 
that  makes  the  figures  a  series,  then  holding 
the  principle  in  memory  while  testing  the 
multiple  choices  for  their  appropriateness. 

How  is  this  construct  related  to  other  constructs/ 
factors:  Factor  analyses  of  experts'  judgments  (Wing, 

Peterson,  &  Hoffman,  1984)  found  figural  reasoning  in 
the  cognitive  rather  than  spatial  factor.  Among  the 
cognitive  constructs,  it  clustered  apart  from  verbal, 
quantitative,  speed,  and  memory.  See  3. A.  below. 
Construct  Validity. 

B.  Taxonomy/types/categories  of  items: 

How  the  test  is  scored/scaled:  Total  number  correct 

How  are  item  type  and  difficulty  jointly  sampled: 

To  be  determined  in  the  next  round  of  validation  by 
examining  item  specifications  and  data. 

C.  Seeming  differences  across  subgroups  in  experiences 
relevant  to  any  type  of  item  or  in  offensiveness  of 
content  (e.g.,  experience  with  computers): 

Probable  SES  related  differences  in  exposure  to 
posers  and  diagrammatic  stimuli. 

D.  Basis  for  expecting  incremental  validity  in  specific 
jobs:  The  panel  of  expert  judges  in  Project  A  estimated 
that  figural  reasoning  would  predict  success  on  numerous 
tasks,  like  troubleshooting,  inspecting,  and  repairing 
various  types  of  systems  (e.g.,  mechanical,  electrical, 
fluid),  detecting  and  identifying  targets,  and  decoding 
and  analyzing  data.  Estimated  validities  ranged  from  .18 
to  .41,  centering  around  .3  for  the  tasks  just  cited. 

2.  Precision  of  measurement 

A.  Test's  consistency  across 
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Different  forms:  The  test  has  one  form  that  exists  on 
two  media:  P/P  and  software.  To  date  these  alternate 
media  of  the  same  form  have  not  been  administered  to 
the  same  examinees. 

Testing  occasions:  .65  in  >  468  incumbents 

Levels  of  total  test  score  (score  conditional  SEM) :  To 
be  determined  under  the  follow  on  contract  to  Project  A. 

B.  For  power  or  speeded  test:  internal  consistency: 
Split-half  (odd-even,  Spearman-Brown)  reliability  of  .87 

C.  Variation  across  subgroups  in  score-conditional  SEM:  To 
be  determined  under  the  follow  on  contract  to  Project  A 

D.  If  score  is  model-based,  what's  the  model-based  precision 
N.  A. 

E.  Subgroup  differences  in  item  functioning: 

Fairness  analyses  will  be  carried  out  under  the  contract 
that  follows  up  Project  A.  Means  and  s.d.'s  of  the 
number  correct  measure  by  group  in  the  Concurrent 
Validation  were  as  follows: 


Group  n 

X 

s.d. 

Whites  6,002 

20.41 

5.15 

Blacks  2,547 

16.34 

5.64 

Hispanics  333 

17.39 

6.06 

Other  354 

18.26 

5.99 

Females  885 

19.95 

4.76 

Males  8,351 

19.01 

5.74 

3 .  Validity 

A.  Construct  validity:  correlations  with  other  tests  of 
similar  things.  Intercorr.  with  ASVAB: 

In  the  field  test  of  the  Project  A  battery,  a  pilot 
version  of  Reasoning  correlated  .47  and  .74, 
respectively,  with  marker  tests  SRA  Word  Grouping  and 
DAT  Tdjstract  Reasoning,  which  were  administered 
concurrently  to  the  118  Army  examinees  (Peterson,  1987). 
The  verbal  nature  of  the  stimuli  in  the  first  test  was 
assumed  to  account  for  its  lower  correlation. 

According  to  new  data  from  the  Navy  (Wolfe,,  unpublished, 
1989),  Reasoning  correlates  .655  with  Ravens  and  .630 
with  a  composite  of  ASVAB 's  power  tests  in  a  sample  of 
567  Navy  recruits. 

In  factor  analyses  of  the  Concurrent  Validation  data, 
both  with  and  without  ASVAB,  Reasoning  loaded  most 
heavily  on  the  spatial  factor  and  had  a  moderate  loading 
on  the  factor  formed  by  the  power  tests  of  ASVAB  (pp.  81 
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-  84) .  Its  highest  correlations  among  ASVAB  subtests  are 
with  AR  (.51),  MK  (.47),  and  MC  (.46).  The  squared 
multiple  R  with  ASVAB  was  .34. 

B.  Uniqueness  re  ASVAB.  How  assessed?  Its  uniqueness 
is  .53  for  internal  consistency  and  .44  for  retest 
reliability  in  9,332  incumbents. 

C.  Incremental  validity  re  ASVAB?  Improvement  in  SD  of  the 
redicted  criterion?  How  many  other  tests  were  considered 
when  this  one  was  validated?  In  Marine  JPM  data  to  date, 
the  Reasoning  Test  adds  .031  in  explained  variance  in 
knowledge  test  scores  and  .027  in  supervisors'  ratings  in 
>1,100  Marines  (Mayberry,  unpublished,  1989). 

In  the  1987  sample  of  TOW  gunners  (p.  100) ,  Reasoning  was 
not  significant  in  any  of  the  GLM  analyses.  Its 
simple  correlation  with  accuracy  on  Event  5  was  .151 
(p  =  .0050),  compared  with  .179  for  the  Combat  Composite 
(AR+CS+MC+AS)  and  .145  for  AFQT. 

D.  Subgroup  differences  in  incremental  validity  or  in  std 
err  of  prediction  of  the  criterion?  To  be  calculated 
under  the  contract  following  up  Project  A. 

E.  Job  families  and  patterns  of  validities  for  them. 
Similarity  of  job  families  in  the  other  Services. 

Expected  utility  in  terms  of  numbers,  attrition,  training 
costs,  attractiveness  to  applicants:  There  is  a  great 
number  of  small  fill  MOS  that  do  inspection,  trouble¬ 
shooting,  and  repair  of  various  systems:  electrical, 
electronic,  mechanical,  and  fluid.  Presumably  these 

MOS  have  counterparts  in  all  of  the  other  Services. 

It  bears  repeating,  however,  that  measures  of  many 
predictor  constructs  will  relate  to  specific  tasks  much 
better  than  to  whole  jobs.  If  Reasoning  is  a  measure  of 
fluid  intelligence,  there  is  some  question  as  to  whether 
the  available  performance  measures  are  appropriate  as 
criteria. 

See  above  p.  6  -  7  for  the  validities  and  incremental 
validities  of  a  composite  of  Project  A  "spatial" 
predictors  for  nine  large  fill  MOS. 

4 .  Equating 

A.  Problems  in  developing  equivalent  forms  (e.g..  How  large 
is  the  potential  item  pool?  Would  pretesting  and  vetting 
of  new  items  be  needed?  Could  item  devel  be  automated?) 
The  potential  item  pool  is  unlimited.  Pretesting  would 
be  important  as  item  statistics  cannot  easily  be 
determined  rationally.  Automated?  Perhaps  in  part,  but 
a  great  deal  of  judgment  and  adjustment  of  the  products 
would  probably  be  needed. 
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B>  Types  of  samples  needed/acceptable  foir  inifial  equating? 
Testing  conditions  needed?  Adequate  range  of  abilities. 
Sufficient  Ns  in  the  small  cells. 

C.  Subgroup  dependence  of  equatings:  To  be  calculated  in 
pre-implementation  research. 

D.  Practicality  of  pencil -and-paper  versions;  The  present 
form  is  pencil-and-paper. 

5.  Feasibility  for  operational  S/C 

A.  Min. /Max.  admin  times.  Times  for  instrux/test  for  low 
and  high  AFQT  examinees:  12  minutes  is  allowed  for  doing 
the  items.  Instruction  time  is  about  5  min.  more  in  the 
setting  of  a  group  test.  NPRDC  has  computerized  this 
test.  Their  software  permits  examinees  to  skip 
questions,  presenting  them  again  at  the  end  of  the  test, 
if  time  allows.  Dave  Alderton  of  NPRDC  reports  that  the 
computerized  version  of  Reasoning  has  shown  the 
following  time  characteristics  in  a  sample  of  1,305  Navy 
recruits  recently; 

ComponentX 
Instructions 
Items 
Total 

B.  Susceptibility  to; 

Cheating,  compromise:  Probably  low 

Irrelevant  response  strategies,  coaching:  Probably  low 

Practice;  On  retesting,  500  incumbents  gained  an  average 
of  .27  s.d.,  from  19.16  to  20.64  out  of  36. 

C.  Floor/ceiling  effects?  Performance  of  low/high  AFQTs 
on  the  test?;  Floor/ ceiling  effects  not  apparent. 
Performance  by  AFQT  level  to  be  calculated  in  the 
contract  that  follows  up  Project  A. 

D.  Examinees'  reactions;  motivation,  enjoyment,  boredom: 
Nothing  noteworthy 

E.  Limitations  of  CAT-ASVAB  hardware  for  delivering  the 
test.  Additional  hardware  required:  None;  it's  already 
up  at  NPRDC  on  the  CAT  system. 

F.  Need  for  special  eqpt  with  special  maintenance?  None 


mean _ s.d. 

2.59  min  1.08 
9.33  2.25 

11.92  2.73 


maximum  time 
12.37 
12.08 
21.63 
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COMPUTERIZED  TESTS  OF  PSYCHOMOTOR,  PERCEPTUAL, 

AND  MEMORIAL  ABILITIES  (pp.  41  -  80) 

The  computerized  battery  from  Project  A  is  a  diverse  set  of 
tests  that  are  administered  on  a  portable  computer.  Stimuli  and 
self-paced  directions  are  shown  on  the  computer's  video  screen, 
and  the  examinee  responds  on  a  special  response  device  (p.  40) . 
By  pressing  buttons  on  the  response  device  the  examinee 
registers  speed  and  choice  of  response;  joy  sticks  and  sliding 
controls  let  the  examinee  control  the  movement  of  a  crosshair  on 
the  screen.  A  dial  is  used  to  indicate  the  hand,  and  thus  the 
joystick,  used  for  two  of  the  tests. 

For  measuring  reaction  time,  the  stimuli  are  not  presented 
until  the  examinee  is  pressing  all  four  "home"  buttons.  That 
constraint  insures  that  examinees  start  pressing  buttons  from  a 
standard  hand  position. 

Different  tests  yield  different  scores,  the  variety  of 
scores  including  reaction  time  as  decision  and  movement  speed, 
accuracy  as  proportion  correct,  accuracy  as  distance  off  target, 
and  intervals  between  the  actual  and  effective  times  to  respond. 

The  following  eight  computerized  tests  are  proposed: 

Cannon  Shoot  Number  Memory  Short-term  Memory 

Two-hand  Tracking  Target  Identification  Target  Shoot 

One-hand-tracking 


To  date,  validities  have  been  reported  in  Project  A  only 
for  a  composite  of  the  computerized  predictors,  not  for  the  indi¬ 
vidual  ability  tests.  That  composite  includes  scores  from  all  of 
the  tests.  The  "computer  composites"  in  the  table  below  are  six 
sets  of  non-overlapping  scores  from  the  new  predictors.  Those 

Project  A  Concurrent  Validation 
Validities  Against  Hands-on  Performance  in  Nine  Major  MOS 
For  ASVAB  and  Computerized  Predictor  Composites 


Predictor 

Composites 

IIB 

13B 

19E 

ASVAB  (4  fac¬ 
tors,  unit  wtd) 
All  computer 

.44 

.22 

.40 

composites 
(unit  wtd) 
Computer  + 

.42 

.23 

.29 

ASVAB 

.49 

.25 

.41 

Source:  McHenry, 

Szenas, 

Occupations 


31C 

63B 

64C 

71L 

91A 

95B 

ALL 

.65 

.30 

.54 

.54 

.58 

.59 

.47 

.52 

.25 

.43 

.51 

.51 

.44 

.40 

.64 

.34 

.54 

.57 

.58 

.60 

.49 

&  Wise  (1987,  unpublished) 
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Names  of  the  occupations  in  the  table  above  are: 


IIB 

Infantryman 

64C 

13B 

Cannon  crewman 

19E 

Armor  crewman 

71L 

31C 

Radio  teletype  operator 

63B 

Light-wheel  vehicle 

91A 

mechanic 

95B 

Motor  transport 
operator 
Administrative 
specialist 
Medical  specialist 
Military  police 


six  composites  are  based  on  the  analysis  on  pp  81  “  85,  but 
Complex  Perceptual  Speed  and  Accuracy  is  broken  into  two 
predictors ,  one  each  for  speed  and  accuracy . 

Separate  and  incremental  validities  for  a  unit  weighted 
composite  of  those  smaller  composites  are  given  in  the  above 
table.  These  aggregate  analyses  are  not  repeated  separately  for 
each  test  at  l.D  and  3.C  of  the  TASP  criteria.  The  criterion 
measure  is  a  composite  of  MOS-specific  hands-on  tasks. 

In  the  interests  of  evaluating  end-of -training  measures, 
which  are  likely  to  be  written  tests,  the  validities  of  Project 
A's  computerized  tests  for  predicting  written  measures  are  shown 
in  the  next  table.  The  predictor  is  a  composite  of  the  new 
computerized  tests;  the  criterion  is  a  composite  of  end-of- 
training  and  on-the-job  knowledge  tests  in  the  CV.  Unlike  end— 
of-training  tests  that  are  a  part  of  training  courses,  the 
Project  A  knowledge  tests  were  designed  to  have  ample  variances. 
Thus,  the  correlations  in  this  next  table  are  probably  an 
optimistic  estimate  of  the  validities  to  be  expected  in  the  ECAT 
validation. 


Project  A  Concurrent  Validation 
Validities  Against  Knowledge  Tests  in  Nine  Major  MOS 
For  ASVAB  and  Computerized  Predictor  Composites 


Predictor 

Composites 

IIB 

13B 

19E 

31C 

63B 

64C 

71L 

91A 

95B 

ALL 

ASVAB  (4  fac¬ 
tors,  unit  wtd) 

.77 

.61 

.76 

.77 

.80 

.74 

.73 

.81 

.78 

.75 

All  computer 
composites 
(unit  wtd) 
Computer  + 

.65 

.59 

.69 

.68 

.59 

.63 

.65 

.66 

.67 

.65 

ASVAB 

.78 

.64 

.78 

.77 

.80 

.77 

.75 

.81 

.79 

.77 

Source:  McHenry,  Szenas,  &  Wise  (1987,  unpublished) 
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Response  device  for  Project  A  computerized  predictors 


The  case  for  Cannon  Shoot  to  be  in  the  ECAT  Validation 


Description: 


Cannon  Shoot  is  a  36-itein  test  from  the 
computerized  battery  of  Project  A  that  involves 
estimating  the  time  to  fire  a  projectile  from  a 
static  cannon  so  as  to  hit  a  target  moving  across 
the  line  of  fire.  The  target  and  shell  move  in 
straight  lines.  Speed  of  the  shell  is  constant 
across  trials;  target  speed  and  direction  vary 
over  trials.  Practice  items  and  self-paced 
instructions  are  given  on  the  screen.  A  sequence 
of  four  sample  screens  is  given  below. 


o 


A1 


1.  Theoretical  development  of  construct  and  measure (s) 

A.  Definition  of  construct  (process [es] ) 

This  task  involves  judgment  of  rates  of  visible  movement 
and  memory  for  the  rate  of  the  cannon  shell.  Peterson 
(1987)  referred  to  the  construct  as  "estimating  relative 
velocities. " 

How  is  this  construct  related  to  other  constructs/ factors 
See  below  3. A.,  Construct  Validity 

B.  Taxonomy/types/categories  of  items 

Items  vary  in  the  angle  of  target  movement  relative  to 
the  cannon  (12  possible  values) ,  distance  from  the  cannon 
to  the  point  of  impact  (4  possible  values) ,  and  distance 
the  target  travels  from  time  of  firing  to  time  of  direct 
hit  (4  possible  values) . 

How  the  test  is  scored/scaled: 

The  following  scores  have  been  recorded  at  various 
stages  in  Project  A;  mean  minimum  distance  between 
shell  and  target  (raw  and  log  transformation) , 
mean  times  between  actual  and  optimal  firing  (absolute, 
for  early  shots  alone,  and  for  late  shots  alone) ,  and 
proportion  of  hits  (hit  =  shell  passes  within  the 
boundaries  of  the  target) .  Time  measures  are  in 
hundredths  of  a  second;  logs  are  natural  logs. 

How  are  item  type  and  difficulty  jointly  sampled? 

Items'  design  parameters  all  affect  their  difficulty. 

For  the  field  test,  a  Latin  s(^are  design  was  used  to 
generate  48  items  in  the  matrix  of  possible  combinations 
of  parameter  values.  The  field  test  battery  was  reduced 
by  12  items.  In  the  remaining  36  some  are  easy  enough 
to  have  little  variance,  but  the  distribution  of  diffi¬ 
culty  across  items  remains  to  be  checked. 

C-  Seeming  differences  across  subgroups  in  experiences 
relevant  to  any  type  of  item  or  in  offensiveness  of 
content  (e.g.,  experience  with  computers):  In  the 
field  test  (p.  ),  a  correlation  of  .18  (p  <  .05) 

between  test  scores  and  self-reports  of  experience  with 
video  games  was  found. 

D.  Basis  for  expecting  incremental  validity  in  specific 
jobs.  It  appears  to  researchers  in  Project  A  that 
judgment  of  movement  is  involved  in  visually  guided 
gunnery,  driving  (trucks,  tanks,  and  construction 
equipment),  air  traffic  control,  and  calling  for  fire  on 
moving  targets.  ASVAB,  of  course,  does  not  measure 
this  construct.  This  construct  was  not  included  in  the 
expert  judgment  task  which  guided  the  choice  of 
predictors  in  Project  A. 
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2.  Precision  of  measurement 


A.  Test's  consistency  across 

Different  forms:  The  test  has  one  form  at  present. 

Testing  occasions:  For  the  measure  of  mean  absolute  time 
between  actual  and  optimal  firing,  reliability  over  an 
interval  of  two  weeks  was  .52  in  480  incumbents.  An 
alternative  scoring  method,  one  that  discards  items  which 
have  low  variance,  is  to  be  evaluated  soon  in  an  effort 
to  improve  retest  reliability. 

Levels  of  total  test  score  (score  conditional  SEM) : 

To  be  determined  in  the  follow  on  contract  to  Project  A. 

B.  For  power  or  speeded  test:  internal  consistency:  An 
odd-even  reliability  of  .65  was  observed  on  the  9,000+ 
cases  in  the  CV. 

C.  Variation  across  subgroups  in  score-conditional  SEM: 

To  be  determined  in  the  follow  on  contract  to  Project  A. 

D.  If  score  is  model-based,  what's  the  model -based  precision 
N.A. 

E.  Subgroup  differences  in  item  functioning: 

Fairness  analyses  will  be  carried  out  under  the  contract 
that  follows  up  Project  A.  Means  and  s.d.'s  of  the 
latency  measure  (mean  discrepancy  between  optimal  and 
actual  time  to  fire)  by  group  in  the  Concurrent 
Validation  were  as  follows: 


Group 

n 

X 

S.d. 

Whites 

5,992 

4.93 

13.52 

Blacks 

2,542 

7.23 

15.92 

Hispanics 

334 

4.46 

13.81 

Other 

354 

4.96 

12.85 

Females 

885 

5.52 

18.33 

Males 

8,337 

5.55 

13.74 

3.  Validity 

A.  Construct  validity:  correlations  with  other  tests  of 
similar  things.  Intercorr.  with  ASVAB:  Although  it 
involves  minimal  motion  (viz.,  pressing  a  button  that 
the  hand  is  poised  over).  Cannon  Shoot's  mean  time 
discrepancy  loads  most  heavily  with  psychomotor  scores 
(pp.  81  -  84) .  The  highest  correlation  with  an 
ASVAB  subtest  is  .30  for  MC,  while  the  median  correlation 
with  ASVAB  subtests  is  .19  (p.  87).  Squared  multiple  R 
with  all  ASVAB  subtests  is  .09  for  that  same  score. 
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B.  Uniqueness  re  ASVAB.  How  assessed?  Uniqueness  relative 
to  all  ASVAB  subtests  is  estimated  at  .56  and  .43  for 
the  mean  absolute  time  discrepancy  score,  using  internal 
consistency  and  retest  reliabilities,  respectively. 

C.  Incremental  validity  re  ASVAB?  Improvement  in  SD  of  the 
predicted  criterion?  How  many  other  tests  were 
considered  when  this  one  was  validated?  Incremental 
validities  of  all  computerized  and  spatial  predictors 
from  Project  A  were  examined  in  GLM  analyses  by  Silva 
(1989,  unpublished).  The  sample  was  311  trainees  who 
had  taken  ASVAB  as  applicants,  taken  the  Project  A  tests 
during  their  first  three  days  of  Service,  and  then  been 
tested  on  their  anti-tank  gunnery  skills  about  twelve 
weeks  later.  The  criterion  measure  was  an  accuracy  score 
on  the  fifth  event  (i.e.,  set  of  10  trials),  which  was 
the  first  event  on  which  a  gunner  could  attain  qualifi 
cation.  Correlations  of  all  Project  A  tests,  AFQT,  and 
ASVAB 's  Combat  Composite  with  the  criterion  were  first 
examined.  Variance  in  the  criterion  accounted  for  by 
two  sets  of  tests  -  all  Project  A  and  all  ASVAB  -  was 
computed  separately  and  then  same  analysis  was  run  on  the 
Project  A  tests  with  the  criterion  variance  associated 
with  ASVAB  removed.  Finally,  the  Project  A  tests  were 
entered  into  a  stepwise  linear  regression  on  the  residual 
(unrelated  to  ASVAB)  variance  (p.  100) . 

Cannon  Shoot  correlated  .159  (.0029)  with  the  criterion, 
compared  with  .179  for  the  Combat  Composite  (CS  +  AR  +  MC 
+  AS)  and  .145  with  AFQT.  The  multiple  for  all  ASVAB 
subtests  was  .041,  for  all  Project  A  was  .174,  and  for 
both  sets  together  .181.  Project  A  tests  accounted  for 
.133  of  the  residual  variance.  In  the  various  GLM 
analyses.  Cannon  Shoot  did  not  account  for  significant 
unique  variance,  nor  enter  into  the  stepwise  regression. 

D.  Subgroup  differences  in  incremental  validity  or  in  std 
err  of  prediction  of  the  criterion?  To  be  determined 
under  the  contract  that  follows  up  Project  A. 

E.  Job  families  and  patterns  of  validities  for  them. 
Similarity  of  job  families  in  the  other  Services. 

Expected  utility  in  terms  of  n\imbers,  attrition, 
training  costs,  attractiveness  to  applicants: 

Validity  is  more  likely  to  generalize  across  jobs  at 

the  level  of  [conceptually  appropriate]  tasks  than  at  the 
level  of  the  whole  job.  The  gunnery  tasks  requiring 
movement  judgment  are  in  large  fill,  hard  to  fill  MOS. 
Likewise  for  tank  driving.  The  MOS  for  truck  drivers 
has  fairly  large  numbers.  Air  traffic  controllers  are 
not  numerous,  but  utility  of  good  performance  is  high. 

4 .  Equating 
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A.  Problems  in  developing  equivalent  forms  (e.g. ,  How  large 
is  the  potential  item  pool?  Would  pretesting  and  vetting 
of  new  items  be  needed?  Could  item  deyel  be  automated? 

As  there  are  several  continuous  dimensions  of  item 
difficulty,  the  potential  item  pool  is  large.  Equivalent 
forms  could  be  produced  by  merely  scrambling  or  rotating 
the  existing  items.  An  algorithm  to  generate  items 
could  be  written. 

B.  Types  of  samples  needed/accept8d>le  for  initial  equating? 
Testing  conditions  needed? 

Numbers  of  minority  and  low  ASVAB  examinees  will  set 
lower  limits  on  sample  sizes. 

C.  Subgroup  dependence  of  equatings:  To  be  determined. 

D.  Practicality  of  pencil-and-paper  versions 

Nil.  Computerization  makes  this  kind  of  test  possible. 

5.  Feasibility  for  operational  S/C 

A.  Min. /Max.  admin  times.  Times  for  instrux/test  for  low 

and  high  AFQT  examinees:  The  correlation  of  time  to 

read  the  instructions  for  the  whole  computerized  battery 
and  AFQT  is  about  .30.  In  the  CV,  mean  total  test  time 
plus  and  minus  two  standard  deviations  equalled  10m  09sec 
to  3m  33sec. 

B.  Susceptibility  to; 

Cheating,  compromise,  irrelevant  response  strategies: 
Probably  low.  The  latency  variable  is  reckoned  from  the 
optimal  time  to  shoot,  so  there  is  not  a  speed/accuracy 
tradeoff. 

Practice:  On  retesting,  487  incumbents  improved  their 

accuracy  by  .3  s.d.  on  the  time  discrepancy  score  (p.  98) 

Coaching:  Unknown;  doubtful 

C.  Floor/ceiling  effects?  Performance  of  low/high  AFQTs 
on  the  test?  To  be  determined  under  the  contract  that 
follows  up  Project  A 

D.  Examinees'  reactions:  motivation,  enjoyment,  boredom: 
Fun 

E.  Limitations  of  CAT-ASVAB  hardware  for  delivering  the 
test.  Additional  hardware  required:  Any  single  key 
could  serve  to  fire  the  cannon,  so  no  special  response 
device  is  needed. 

F.  Need  for  special  eqpt  with  special  maintenance? 

None. 
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Target  Shoot 


Target  shoot  is  a  30-trial  test  of  psychomotor  abilities 
from  the  computerized  tests  of  Project  A.  The  examinee 
uses  a  joy  stick  to  move  a  crosshair  inside  a  target  that 
moves  unpredictably  on  the  video  screen,  and  then  shoot  the 
target  by  pressing  a  button  with  the  other  hand.  As  soon  as 
the  examinee  fires,  the  trial  ends.  Practice  items  and 
self-paced  instructions  are  given  on  the  screen.  A  sample 
screen  is  shown  below. 
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1.  Theoretical  development  of  construct  and  measure (s) 

A.  Definition  of  construct  (process [es]) 

Domain:  Psychomotor 

This  construct  is  the  ability  to  make  movements  so  as 
to  adjust  a  movable  part  accurately.  For  this  test,  the 
adjustment  is  made  in  response  to  erratic,  rather  than 
predictable,  movements  of  the  stimulus.  Fleishman's 
psychomotor  constructs  of  control  precision  and  rate 
control  are  involved. 

How  is  this  construct  related  to  other  constructs/factors 
Control  precision  is  involved  in  the  1-hand  and  2-hand 
Tracking  tests  as  well,  but  Target  Shoot  involves  the 
added  ability  of  adjusting  to  unpredictable  movement  in 
the  stimulus,  which  Fleishman  defines  as  Rate  Control. 

In  addition,  the  examinee  must  make  a  gross  movement  with 
the  other  hand  at  just  the  right  moment,  so  as  to  hit  the 
target.  Precision  in  "firing"  may  involve  aiming 
ability,  quickness  of  reaction,  and/or  multi-limb 
coordination,  all  of  which  are  constructs  of  Fleishman's. 

B.  Taxonomy/types/categories  of  items: 

Items  vary  in  difficulty  with  average  target  speed, 
maximum  crosshair  speed,  and  frequency  of  changes 
in  course  and  speed  by  the  target. 

How  the  test  is  scored/scaled? 

The  main  measure  of  interest  is  Mean  Log  (Distance  + 

1),  which  is  distance  off  target  at  the  moment  of 
firing,  percent  hits,  and  time  to  fire. 

How  are  item  type  and  difficulty  jointly  sampled? 

This  question  will  be  answered  as  soon  as  we  can 
examine  the  item  specifications  and  data. 

C.  Seeming  differences  across  subgroups  in  e^eriences 
relevant  to  any  type  of  item  or  in  offensiveness  of 
content  (e.g.,  experience  with  computers): 

Self  reported  experience  with  video  games  correlated 
.27  (p  «  .05)  with  Target  Shoot  for  256  incumbents 
in  the  field  test  of  Project  A.  Such  experience  may 
related  to  SES.  Content  would  offend  only  those  who 
disapprove  of  shooting. 

D.  Basis  for  expecting  incremental  validity  in  specific  jobs 

-  Face  validity:  this  test  resembles  in  two  dimensions 
a  real  gunnery  task 

-  The  judgment  of  experts  in  Project  A  (Wing,  Peterson, 
and  Hoffman,  1984)  was  that  measures  of  rate  control 
and  precision  control  would  be  good  predictors  of  all 
kinds  of  gunnery:  heavy  direct  fire  weapons. 
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individual  weapons,  and  indirect  fire. 

-  ASVAB  does  not  have  psychomotor  tests;  and  psychomotor 
and  general  cognitive  abilities  have  a  relatively  low 
correlation 

2.  Precision  of  measurement: 

A.  Test's  consistency  across: 

Different  forms:  Target  shoot  has  one  form  at  present. 

Testing  occasions:  Retest  reliabilities  of  .37  and  .58 
were  found  for  the  error  and  time  to  fire  scores,  in 
order,  in  480  incumbents. 

Levels  of  total  test  score  (score  conditional  SEM) : 

To  be  computed  in  the  follow-on  to  Project  A. 

B.  For  power  or  speeded  test,  internal  consistency:  The 
odd-even  reliabilities  of  the  error  and  time  to  fire 
measures  were  .74  and  .85  in  9,000  incumbents. 

C.  Variation  across  subgroups  in  score-conditional  SEM: 

To  be  calculated  in  the  follow  on  to  Project  A. 

D.  If  score  is  model-based,  what's  the  model-based  precision 
N.  A. 

E.  Subgroup  differences  in  item  functioning: 

Fairness  analyses  will  be  carried  out  under  the  contract 
that  follows  up  Project  A.  Means  and  s.d.'s  of  the 
error  measure,  mean  log  (distance  off  target  at  moment  of 
firing,  +1) ,  by  group  in  the  Concurrent  Validation  were 
as  follows: 


Group 

n 

X 

s.d. 

Whites 

5,870 

2.16 

0.23 

Blacks 

2,346 

2.22 

0.26 

Hispanics 

324 

2.15 

0.20 

Other 

340 

2.18 

0.26 

Females 

774 

2.31 

0.29 

Males 

8,106 

2.16 

0.23 

3 .  Validity 

A.  Construct  validity:  correlations  with  other  tests  of 
similar  things.  Intercorrelations  with  ASVAB: 

In  a  factor  analysis  with  pre-enlistment  ASVAB  subtests 
and  the  Project  A  spatial  and  computerized  predictors  (p. 
81) ,  the  error  and  time  measures  loaded  most  heavily  on 
the  psychomotor  factor.  Other  factor  analyses  with  fewer 
predictors  provide  similar  clustering  and  loadings  (pp. 

82  -  4).  The  loadings  on  the  psychomotor  factor  are 
moderate,  and  no  strong  secondary  loadings  are  present. 
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Target  shoot  has  low  correlations  with  all  ASVAB  subtests 
(median  <  .14,  max.  .23)  (p.  90),  and  a  squared  multiple 

correlation  with  all  the  subtests  <  .10  (p.  95). 

B.  Uniqueness  re  ASVAB.  How  assessed?  Estimated  at  .70  and 
.37  for  the  error  score  with  respect  to  internal 
consistency  and  retest  reliability. 

C.  Incremental  validity  re  ASVAB?  Improvement  in  SD  of  the 

predicted  criterion?  How  many  other  tests  were 
considered  when  this  one  was  validated?  Incremental 

validities  of  all  computerized  and  spatial  predictors 
from  Project  A  were  examined  in  GIM  analyses  by  Silva 
(1989,  unpublished).  The  sample  was  311  trainees  who 
had  taken  ASVAB  as  applicants,  taken  the  Project  A  tests 
during  their  first  three  days  of  Service,  and  then  been 
tested  on  their  anti-tank  gunnery  skills  about  twelve 
weeks  later.  The  criterion  measure  was  an  accuracy  score 
on  the  fifth  event  (i.e.,  set  of  10  trials) ,  which  was 
the  first  event  on  which  a  gunner  could  attain 
qualification.  Correlations  of  all  Project  A  tests, 

AFQT,  and  ASVAB 's  Combat  Composite  with  the  criterion 
were  first  examined.  Variance  in  the  criterion 
accounted  for  by  two  sets  of  tests  -  all  Project  A  and 
all  ASVAB  -  were  was  computed  separately  and  then  same 
analysis  was  run  on  the  Project  A  tests  with  the 
criterion  variance  associated  with  ASVAB  removed. 

Finally,  the  Project  A  tests  were  entered  into  a 
stepwise  linear  regression  on  the  residual  (unrelated  to 
ASVAB)  variance  (p.  100) . 

Target  shoot  correlated  .112  with  the  criterion,  compared 
with  .179  for  the  Combat  Composite  and  .145  with  AFQT. 

The  multiple  for  all  ASVAB  subtests  was  .041,  for  all 
Project  A  was  .166,  and  for  both  sets  together  .177.  In 
predicting  the  residual  criterion  variance.  Target  Shoot 
did  not  contribute  unique  variance,  and  it  did  not  enter 
significantly  into  the  stepwise  regression. 

D.  Subgroup  differences  in  incremental  validity  or  in  std 
err  of  prediction  of  the  criterion?  To  be  computed  in 
the  follow-on  to  Project  A. 

E.  Job  families  and  patterns  of  validities  for  them 
Similarity  of  job  families  in  other  Services. 

Expected  utility  in  terms  of  numbers,  attrition,  training 
costs,  attractiveness  to  applicants;  The  job  family  or 
whole  single  job  is  probably  too  gross  a  unit  of  job 
analysis  to  display  a  single  test's  potential  for  helping 
with  classification.  Target  Shoot  is  expected  to  show 
gains  in  validity  against  gunnery  tasks,  which  are 
critical  in  the  large  fill,  hard  to  fill  combat  MOS. 
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4 .  Equating 

A.  Problems  in  developing  equivalent  forms  (e.g..  How  large 
is  the  potential  item  pool?  Would  pretesting  and  vetting 
of  new  items  be  needed?  Could  item  devel  be  automated?) 
The  potential  item  pool  is  very  large.  Other  forms  could 
be  created  by  scrambling  the  order  and/or  spatial 
orientation  of  the  present  items.  Items  probably  could  be 
created  by  an  algorithm,  but  writing  the  algorithm  would 
cost  something.  Would  pretesting  ever  not  be  advisable? 

B.  Types  of  samples  needed/acceptable  for  initial  equating? 
Samples  with  adequate  numbers  of  the  small  subgroups 

Testing  conditions  needed?  Suitable  for  testing  in 
general  and  computerized  testing  in  particular.  Nothing 
unusual 

C.  Subgroup  dependence  of  equatings:  To  be  addressed  when 
equating  reached. 

D.  Practicality  of  pencil-and-paper  versions:  Zero;  this 
kind  of  test  is  possible  because  of  computers. 

5.  Feasibility  for  operational  S/C; 

A.  Min. /Max.  admin  times.  Times  for  instrux/test  for  low 
and  high  AFQT  excuninees:  Mean  total  test  time  was  4m  50s 
in  9,208  incumbents  (s.d.,  30s).  Times  have  not  been 
broken  out  by  AFQT  level,  but  the  correlation  of  time  to 
read  the  directions  on  the  computerized  tests  in  Project 
A  with  AFQT  is  about  .30. 

B.  Susceptibility  to: 

Cheating,  compromise:  Given  the  large  potential  set  of 
items,  the  danger  here  may  be  small. 

Irrelevant  response  strategies;  To  counter  a  strategy  of 
sacrificing  accuracy  for  speed,  either  use  an  accuracy 
score,  or  develop  a  composite  predictor  that  adjusts 
speed  of  firing  for  miss  distance.  In  principle,  an 
examinee  could  try  to  hit  near  the  target,  instead  of 
right  inside.  Whether  that  strategy  would  affect  the 
rank  order  of  scores  is  not  yet  known. 

Coaching:  Unknown;  not  obviously  large. 

Practice:  Gains  on  a  first  retesting  were  negligible  in 

>  472  incumbents  (p.  96) . 

C.  Floor/ceiling  effects?  Performance  of  low/high  AFQTs 
on  the  test?  Variability  is  great  enough  to  produce 
significant  correlations.  Easiness  is  not  a  problem: 
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the  mean  log  (distance  +  1)  is  about  9  s.d.  above  the 
theoretical  perfect  level,  and  mean  time  to  fire  is 
about  5  s.d.  above  instantaneous  firing. 

D.  Examinees*  reactions:  motivation,  enjoyment,  boredom: 
This  test  is  fun. 

E.  Limitations  of  CAT-ASVAB  hardware  for  delivering  tte 
test.  Additional  hardware  required:  Something  like  the 
Project  A  response  pedestal  would  be  needed:  at  least 
two  joy  sticks  and  two  buttons  (for  selecting  the  right 
or  left  joy  stick) . 

F.  Need  for  special  eqpt  with  special  maintenance? 

None  other  than  that  above 
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The  case  for  One-hand  Tracking  to  be  in  the  ECAT  Validation 

Description:  One-hand  tracking  is  an  IS-item  test  of  eye-hand 

coordination  from  the  computerized  battery  of 
Project  A.  The  examinee  uses  a  joy  stick  to  try  to 
keep  a  crosshair  inside  a  square  as  it  moves  along 
a  path.  The  path,  which  is  explicitly  marked  on 
the  video  screen,  consists  of  straight  horizontal 
and  vertical  segments.  Practice  items  and  self- 
paced  instructions  are  given  on  the  screen.  A 
sample  screen  is  shown  below. 
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Theoretical  development  of  construct  and  measure (s) 

A.  Definition  of  construct  (process [es] ) : 

This  test  measures  Fleishman's  precision  of  control  in 
the  psychomotor  domain.  Control  precision  is  "the 
ability  to  make  fine,  highly  controlled  movements  to 
adjust  a  machine  control  mechanism  in  response  to  a 
stimulus  whose  speed  and  direction  of  movement  are 
perfectly  predictable,  (p.  5-39)"  (Peterson,  1987) 

How  is  this  construct  related  to  other  constructs  or 
factors?  Tasks  involving  the  ability  of  multi-limb 
coordination  may,  like  the  Two-hand  Tracking  test  in 
Project  A,  also  involve  control  precision.  Fleishman's 
factor  of  rate  control  is  defined  in  part  by  the 
unpredictable  path  of  the  target,  so  it  is  not  involved 
here. 

B.  Taxonomy/types/categories  of  items: 

Items  vary  in  difficulty  with  maximum  speed  of  the  cross¬ 
hairs,  difference  between  that  speed  and  target  speed, 
and  number  of  turns.  Target  speed  and  total  length  of 
path  are  controlled  together  so  that  the  total  time  for 
each  item,  and  thus  the  number  of  measurements  of 
distance  off  target,  would  be  equal  for  all  items. 

How  the  test  is  scored/scaled?  The  one  score  which 
has  been  used  for  analyses  to  date  is  Mean  Log 
(Distance  off  Target  +  1) 

How  are  item  type  and  difficulty  jointly  S2unpled? 

The  item  parameters  are  crossed  to  produce  items  of 
varying  difficulty,  then  the  sequence  of  items  is 
determined  by  randomly  sampling  the  matrix.  This 
procedure  was  chosen  on  the  basis  of  evidence  in 
pilot  tests  that  there  was  little  practice  effect 
within  a  testing  session. 

C.  Seeming  differences  across  subgroups  in  experiences 
relevant  to  any  type  of  item  or  in  offensiveness  of 
content  (e.g.,  experience  with  computers);  Items  are 
content-free,  so  offensiveness  if  not  an  issue.  There 
may  be  SES-related  differences  in  exposure  to  video 
games.  Self-reported  exposure  correlated  .22  with  One- 
hand  Tracking  in  256  incumbents  in  the  Project  A  field 
test  (p  «  .05). 

D.  Basis  for  expecting  incremental  validity  in  specific 

jobs:  In  the  expert  judgment  exercise  early  in  Project 

A,  35  judges  estimated  the  validity  of  control  precision 
against  gunnery  tasks  -  direct  fire  of  heavy  weapons, 
indirect  fire,  and  use  of  individual  weapons  -  to  be 

.25  to  .29.  ASVAB  is  conceptually  and  statistically 
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independent  of  psychoTnotor  measures,  so  the  potential 
for  incremental  validity  is  good. 

2.  Precision  of  measurement 

A.  Test's  consistency  across 

Different  forms:  One  form  of  the  test  exists  now.  The 
intercorrelation  of  One-  and  Two-hand  Tracking  give  a 
rough  estimate  of  alternate  form  reliability  of  One- 
hand  Tracking,  since  both  use  the  same  screens.  That 
figure  ranges  from  .77  in  the  Concurrent  Validation  of 
Project  A  in  1985  to  .81  in  both  TOW  gunnery  trainees 
and  tank  crew  trainees  in  1988. 

Testing  occasions:  Retest  reliability  was  .74  in  468 
first  tour  soldiers. 

Levels  of  total  test  score  (score  conditional  SEM) : 

To  be  computed  under  the  follow  on  contract  to  Project  A. 

B.  For  power  or  speeded  test:  internal  consistency: 

Odd-even  reliability  is  .98  in  the  CV  sample  of  9,000+ 

C.  Variation  across  subgroups  in  score-conditional  SEM: 

To  be  computed  under  the  follow  on  contract  to  Project  A. 

D.  If  score  is  model-based,  the  model-based  precision: 

N.  A. 

E.  Subgroup  differences  in  item  functioning: 

Fairness  analyses  will  be  carried  out  under  the  contract 
that  follows  up  Project  A.  Means  and  s.d.'s  of  the 
error  (distance)  measure  by  group  in  the  Concurrent 
Validation  were  as  follows; 


Group  n 

X 

s.d. 

Whites  5,999 

2.87 

0.44 

Blacks  2,550 

3.23 

0.50 

Hispanics  336 

3.02 

0.48 

Other  354 

3.04 

0.52 

Females  883 

3.39 

0.46 

Males  8,356 

2.94 

0.47 

3 .  Validity 

A.  Construct  validity:  correlations  with  other  tests  of 

similar  things-  Intercorr.  with  ASVAB:  As  noted  above, 
the  correlation  of  One-  and  Two-hand  Tracking  is  in  the 
.77  to  .81  range.  In  factor  analyses  of  Project  A  tests 
with  and  without  pre-enlistment  ASVAB  (pp.  81  -  85) ,  One- 
hand  Tracking  anchors  the  psychomotor  factor  and  has  no 
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strong  loadings  elsewhere.  Its  median  intercorrelation 
with  ASVAB  subtests  is  .235,  the  highest  correlations 
being  with  AS  (.30),  GS  (.28)  and  El  (.27). 

B.  Uniqueness  re  ASVAB.  How  assessed?  In  terms  of  internal 
consistency  and  retest  reliability,  uniqueness  is 
estimated  to  be  .82  and  .58,  respectively. 

C.  Incremental  validity  re  ASVAB?  Improvement  in  SD  of  the 
predicted  criterion?  How  many  other  tests  were 
considered  when  this  one  was  validated?  Incremental 
validities  of  all  computerized  and  spatial  predictors 
from  Project  A  were  examined  in  GLM  analyses  by  Silva 
(1989,  unpublished).  The  sample  was  311  trainees  who 
had  taken  ASVAB  as  applicants,  the  Project  A  tests 
during  their  first  three  days  of  Service  (1986/87) ,  and 
then  performance  tests  on  their  anti-tank  gunnery  skills 
about  twelve  weeks  later.  The  criterion  measure  used 
here  was  accuracy  on  the  fifth  event  (event  =  set  of  10 
trials) ,  which  was  the  first  event  on  which  a  gunner 
could  attain  qualification.  Correlations  of  all  Project 
A  tests,  AFQT,  and  ASVAB 's  Combat  Composite  with  the 
criterion  were  first  examined.  Variance  in  the 
criterion  accounted  for  by  two  sets  of  tests  -  all 
Project  A  and  all  ASVAB  -  was  was  computed  separately 
and  then  same  analysis  was  run  on  the  Project  A  tests 
with  the  criterion  variance  associated  with  ASVAB 
removed.  Finally,  the  Project  A  tests  were  entered  into 
a  stepwise  linear  regression  on  the  residual  (unrelated 
to  ASVAB)  variance. 

One-hand  Tracking  correlated  .206  with  the  criterion, 
compared  with  .179  for  the  Combat  Composite  (CS  +  AR  + 

MC  +  AS)  and  .145  with  AFQT.  In  various  GLM  analyses 
with  all  other  Project  A  tests  entered  (p.  100),  One- 
hand  Tracking  did  not  account  for  significant  unique 
variance. 

In  1988,  a  battery  of  four  Project  A  tests  —  One-  and 
Two-hand  Tracking,  Mazes,  and  Orientation  —  was 
installed  in  two  training  bases  for  a  pilot  test  on 
students  in  the  IIH  anti-tank  and  19K  tank  crew  courses. 
In  samples  of  these  students  (pp.  101-2),  the  validity  of 
an  optimally  weighted  combination  of  all  ASVAB  subtests 
against  scores  on  TOW  and  anti-tank  gunnery  simulators 
was  .266  and  .480,  respectively.  One-hand  Tracking  had  a 
validity  of  .290  and  .484.  In  the  1988  samples,  One- 
hand  Tracking  did  not  account  for  significant  unique 
variance.  But  by  itself,  it  accounts  for  .054  of  the 
residualized  variance,  and  combining  it  with  Mazes  (R^  = 
.007)  increases  the  residual  variance  accounted  for  to 
.076. 
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D.  Subgroup  differences  in  incremental  validity  or  in  std 
err  of  prediction  of  the  criterion? 

Against  the  criterion  of  accuracy  in  firing  the  M70  TOW 
simulator  in  1988,  in  SD  units  of  Mean  Log  (Distance  + 

1)  on  the  composite  of  1-  &  2-hand  Tracking: 

-2  -1  0  +1  +2 
SEP  W  (n  =  457)  1.4  0.9  0.6  0.8  1.3 

SEP  B  (n  =  89)  2.9  2.0  2.1  3.2  4.6 

E.  Job  families  and  patterns  of  validities  for  them. 
Similarity  of  job  families  in  the  other  Services. 

Expected  utility  in  terms  of  numbers,  attrition,  training 
costs,  attractiveness  to  applicants: 

TASP  has  agreed  that  the  focus  of  the  new  tests  will 
be  classification.  It  seems  likely  that  the  gains  in 
validity  over  ASVAB  for  any  new  test  will  be  more  obvious 
at  some  level  of  job  analysis  finer  than  the  whole  job. 
For  One-hand  Tracking,  conceptually  appropriate  tasks 
include 

-  Visually  guided  gunnery,  which  is  critical  in 
Infantry,  Tank  Crew,  some  Air  Defense  jobs, 

-  Driving,  which  is  a  separate  job,  and  is  also 
involved  in  Combat  Engineering,  Mechanized 
Infantry,  Tank  Crew 

Infantry  and  Tank  Crew  are  large  fill,  combat  critical 
MOS  which,  because  of  the  lack  of  similar  civilian  jobs, 
are  hard  to  fill.  Army  spends  large  sums  recruiting 
to  fill  those  specialties. 


4 .  Equating 

A.  Problems  in  developing  equivalent  forms  (e.g..  How  large 
is  the  potential  item  pool?  Would  pretesting  and  vetting 
of  new  items  be  needed?  Could  item  development  be 
automated?  Equivalent  forms  would  be  easy  to  develop, 
either  from  the  limitless  population  of  potential  new 
items  or  from  scrambling  existing  items.  Item  develop¬ 
ment  could  be  done  by  algorithm.  There  may  not  be  enough 
items  in  existence  now  to  allow  predicting  empirical 
parameters  of  new  items. 

B.  Types  of  samples  needed/acceptcible  for  initial  equating? 
Testing  conditions  needed?  The  need  for  enough 
minorities  and  low  AFQT  Examinees  would  drive  sample 
size. 

C.  Subgroup  dependence  of  equatings:  To  be  determined  when 
equating  analyses  are  done. 
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D.  Practicality  of  pencil-and-paper  versions:  The  primary 
outcome  measure  is  sampled  every  hundredth  of  a  second, 
so  it  would  be  hard  to  replicate  on  paper  without  fancy 
apparatus . 

5.  Feasibility  for  operational  S/C 

A.  Min. /Max.  admin  times.  Times  for  instrux/test  for  low 
and  high  AFQT  examinees.  Mean  total  time  for 
instructions  plus  items  was  6m  01s  (s.d.  34s)  for  9,200+ 
incumbents  (p.  104).  Times  by  AFQT  TBD  under  the 
contract  that  follows  up  Project  A. 

B.  Susceptibility  to 

Cheating,  compromise:  Probably  low 

Irrelevant  response  strategies:  An  examinee  could  try  to 
keep  the  crosshair  near  the  target  instead  of  following 
the  instruction  to  keep  it  on  the  target.  This  other 
strategy  is  not  "irrelevant”  since  it  would  involve 
tracking.  Whether  it  would  produce  results  that  are 
different  from  the  instructed  strategy  is  an  empirical 
question. 

Coaching:  There  may  be  coachable  but  relevant  response 
strategies  (things  like  "Keep  your  eye  on  the  ball") . 

Practice:  A  gain  of  .24  s.d.  at  retest  was  found  in  480 
incumbents . 

C.  Floor/ceiling  effects?  Performance  of  low/high  AFQTs: 
Mean  distance  off  target  has  a  large  maximum  score. 

Scores  by  AFQT  TBD  in  the  follow-on  to  Project  A. 


D.  Exeuninees*  reactions:  motivation,  enjoyment,  boredom: 
This  test  is  exciting  and  difficult. 

E.  Limitations  of  CAT-ASVAB  hardware  for  delivering  the 
test.  Additional  hardware  required:  Double  digital  to 
analog  game  board,  clock,  and  response  pedestal  with  two 
joy  sticks. 

F.  Meed  for  special  egpt  with  special  maintenance?  The 
buttons  and  controls  on  the  response  pedestal  sometimes 
need  repair  or  replacement. 
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The  case  for  Two-hand  Tracking  to  be  in  the  ECAT  Validation 


Description: 


Two-hand  tracking  is  a  an  IS-item  test  of  multi- 
liinb  coordination  from  the  computerized  tests  of 
Project  A.  The  examinee  uses  two  sliding  controls, 
one  moving  horizontally  and  one  vertically,  to  try 
to  keep  a  crosshair  inside  a  square  as  it  travels 
along  a  path.  The  path,  which  consists  of  straight 
vertical  and  horizontal  segments,  is  explicitly 
marked  on  the  video  screen.  Practice  items  and 
self-paced  instructions  are  given  on  the  screen. 

A  sample  screen  is  shown  below. 
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1.  Theoretical  development  of  construct  and  measure fs) 


A.  Definition  of  construct  f process  Fes 1 ) 

This  test  measures  the  ability  to  coordinate  the 
simultaneous  movement  of  two  limbs.  Because  the 
movement  is  to  adjust  a  crosshair  to  track  a 
target  moving  continuously  on  a  known  path,  the 
ability  of  control  precision  is  involved  as  well. 

How  js  this  construct  related  to  other  constructs/ 
factors;  In  a  factor  analysis  of  experts'  judgments 
of  validities  of  53  predictor  constructs  against  72 
Army  performance  constructs  (Wing,  Peterson,  &  Hoffman, 
1984),  psychomotor  abilities  formed  a  factor  containing 
clusters  of  dexterity,  coordination,  and  precision/ 
steadiness.  The  judged  constructs  forming  the  latter  two 
clusters  include  multi-limb  coordination  and  control 
precision.  For  the  relation  of  scores  on  this  test  to 
scores  on  the  others  others  in  the  Project  A  battery  and 
to  ASVAB,  see  below  Construct  Validaty,  3. A. 

B.  Taxonomv/tvpes/cateaories  of  items; 

Items  vary  in  difficulty  with  maximum  speed  of  the  cross¬ 
hairs,  difference  between  that  speed  and  target  speed, 
and  number  of  turns.  Target  speed  and  total  length  of 
path  are  controlled  together  so  that  the  total  time  for 
each  item,  and  thus  the  number  of  measurements  of 
distance  off  target  would  be  equal  for  all  items. 

How  the  test  is  scored/scaled?  Mean  Log  (Distance  off 
Target  +  1) 

How  are  item  type  and  difficulty  jointly  sampled? 

The  item  parameters  are  crossed  to  produce  items  of 
varying  difficulty,  then  the  sequence  of  items  is 
determined  by  randomly  sampling  the  matrix.  This 
procedure  was  chosen  on  the  basis  of  evidence  in 
pilot  tests  that  there  was  little  practice  effect 
within  a  testing  session. 

C.  Seeming  differences  across  subgroups  in  experiences 
relevant  to  anv  type  of  item  or  in  offensiveness  of 
content  re.a. .  experience  with  computers);  In  the 
field  test  of  Project  A,  self-reports  of  256  incumbents' 
experience  with  video  games  correlated  .16  (p  <  .05) 
with  their  scores  on  Two-hand  Tracking.  SES  may 
relate  to  differences  in  exposure  to  video  games  and 
diagrammatic  graphics.  Items  are  content-free,  so 
offensiveness  should  not  be  an  issue. 

D.  Basis  for  expecting  incremental  validity  in  specific  jobs 

-  ASVAB  does  not  measure  psychomotor  skills 

-  At  least  some  tasks  for  many  jobs  appear  to  require 
psychomotor  skills. 
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-  In  Project  A  (Wing,  Peterson,  &  Hoffman,  1984),  35 
subject  matter  experts  estimated  the  true  validity 
of  multi-limb  coordination  against  nine  psychomotor 
tasks  from  Army  jobs  to  be  a  median  (across  those 
tasks)  of  .26  (range  .18  -  .37). 

2.  Precision  of  measurement 

A.  Test's  consistency  across 

Different  forms:  One  fora  of  the  test  exists  now.  The 
intercorrelation  of  One-  and  Two-hand  Tracking  give  a 
rough  estimate  of  alternate  fora  reliability  of  Two- 
hand  Tracking,  since  both  use  the  same  screens.  That 
figure  ranges  from  .77  in  the  Concurrent  Validation  of 
Project  A  in  1985  to  .81  in  both  TOW  gunnery  trainees 
and  tank  crew  trainees  in  1988. 

Testing  occasions:  Retest  reliability  was  .85  in  487 
incumbents  from  a  mixture  of  MOS 

Levels  of  total  test  score  (score  conditional  SEM) ; 

To  be  computed  under  the  follow  up  to  Project  A. 

B.  For  power  or  speeded  test:  internal  consistency;  Odd- 
even  reliability  of  .98  for  9,200+  incumbents 

c.  Variation  across  subgroups  in  score  conditional  SEM: 


D.  If  score  is  model-based.  what*s  the  model-based 
precision?  N.A. 


E.  Subgroup  differences  in  item  functioning; 

Fairness  analyses  will  be  carried  out  under  the  contract 
that  follows  up  Project  A.  Means  and  s.d. 's  of  the 
distance  off  target  measure  by  group  in  the  Concurrent 
Validation  were  as  follows; 


Group 

n 

X 

s.d. 

Whites 

5,991 

3.56 

0.49 

Blacks 

2,548 

3.99 

0.43 

Hispanics 

336 

3.76 

0.46 

Other 

352 

3.77 

0.51 

Females 

879 

4.11 

0.38 

Males 

8,348 

3.65 

0.51 

3.  Validity 

A.  Construct  validity;  correlations  with  other  tests  of 
similar  things.  Intercorr.  with  ASVAB; 

In  a  factor  analysis  with  pre-enlistment  ASVAB  subtests, 
the  Project  A  spatial  tests,  and  the  other  Project  A 
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computerized  predictors  (p.  81),  2-hand  Tracking  (error 
score)  had  the  second  highest  (after  1-hand  Tracking 
error)  loading  on  the  psychomotor  factor.  The  median  of 
its  intercorrelations  with  ASVAB  subtests  was  .255,  the 
highest  correlations  being  with  MC  (.41),  AS  (.33),  GS 
(.31),  and  El  (.30).  With  the  1 -hand  Tracking  error 
score  the  correlation  was  .77  in  the  Concurrent 
Validation  (p.  90)  and  .81  in  the  1988  analyses  of 
data  from  simulators  of  TOW  gunnery  and  tank  gunnery. 

In  PROC  GLM  analyses  of  the  gunnery  battery  (viz.,  1-  and 
2-hand  Tracking,  Mazes,  and  Orientation)  against  both  TOW 
and  tank  gunnery  (pp.  101-2),  2-hand  tracking  was  the 
single  strongest  predictor  of  gunnery,  and  1-hand 
tracking  did  not  significantly  enter  the  equations  after 
2-hand.  These  results  do  not  support  the  speculation 
that  1-hand  tracking  is  interchangable  with  2-hand  as  a 
predictor. 

B.  unicmeness  re  ASVAB.  How  assessed?  .79,  in  terms  of 
odd-even  reliability  minus  the  .19  squared  multiple  R 
with  all  ASVAB  subtests.  In  terms  of  retest  reliabil¬ 
ity  (.85),  .66. 

c.  increTnental  validity  re  ASVAB?  Improvement  in  SD  of  the 
predicted  criterion?  How  many  other  tests  were 
considered  when  this  one  was  validated?  Incremental 

validities  of  all  computerized  and  spatial  predictors 
from  Project  A  were  examined  in  GLM  analyses  by  Silva 
(1989,  unpublished).  The  sample  was  311  trainees  who 
had  taken  ASVAB  as  applicants,  the  Project  A  tests 
during  their  first  three  days  of  Service  (1986/87) ,  and 
then  performance  tests  on  their  anti-tank  gunnery  skills 
about  twelve  weeks  later.  The  criterion  measure  used 
here  was  accuracy  on  the  fifth  event  (event  =  set  of  10 
trials) ,  which  was  the  first  event  on  which  a  gunner 
could  attain  qualification.  Correlations  of  all  Project 
A  tests,  AFQT,  and  ASVAB 's  Combat  composite  with  the 
criterion  were  first  examined.  Variance  in  the 
criterion  accounted  for  by  two  sets  of  tests  -  all 
Project  A  and  all  ASVAB  -  was  computed  separately  and 
then  same  analysis  was  run  on  the  Project  A  tests 
with  the  criterion  variance  associated  with  ASVAB 
removed.  Finally,  the  Project  A  tests  were  entered  into 
a  stepwise  linear  regression  on  the  residual  (unrelated 
to  ASVAB)  variance.  The  results  are  shown  on  p.  100. 

Two-hand  Tracking  correlated  .294  with  the  criterion, 
compared  with  .179  for  the  Combat  Composite  (CS  +  AR  + 

MC  +  AS)  and  .145  with  AFQT.  The  multiple  R^  for  all 
ASVAB  subtests  was  .041,  for  all  Project  A  was  .165,  and 
for  both  sets  together  .177.  Project  A  tests  accounted 
for  .126  of  the  residual  variance.  In  all  combinations 
of  tests.  Two-hand  Tracking  accounted  for  significant 
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amounts  of  unique  variance,  and  it  entered  first  into  the 
stepwise  regression  (R^  =  ,0725). 

In  analyses  of  tank  and  anti-tank  gunnery  that  include 
all  ASVAB  subtests  and  four  new  Army  tests  (One-  and 
Two-hand  Tracking,  Mazes,  and  Orientation),  Silva  (1989, 
unpublished)  (pp.  101-2)  has  found  that  Two-hand 
tracking  is  the  single  best  predictor  on  several 
grounds:  highest  zero  order  correlation,  significant 

and  largest  unique  variance,  and  first  of  Project  A 
tests  to  enter  stepwise  regressions  predicting  variance 
after  ASVAB-related  variance  is  removed. 

D.  Subgroup  differences  in  incremental  validity  or  in  std 
err  of  prediction  of  the  criterion?  See  above  2.C. 
Against  the  criterion  of  accuracy  in  firing  the  M70  TOW 
simulator  in  1988,  in  SD  units  of  Mean  Log  (Distance  + 

1)  on  the  composite  of  1-  &  2-hand  Tracking: 

-2  -1  0  +1  +2 
SEP  W  (n  =  457)  1.4  0.9  0.6  0.8  1.3 

SEP  B  (n  =  89)  2.9  2.0  2.1  3.2  4.6 

E.  Job  families  and  patterns  of  validities  for  them. 
Similarity  of  job  families  in  the  other  Services. 

Expected  utility  in  terms  of  numbers,  attrition, 
training  costs,  attractiveness  to  applicants: 

TASP  has  agreed  that  the  focus  of  the  new  tests  will 
be  classification.  It  seems  likely  that  the  gains  in 
validity  over  ASVAB  for  any  new  test  will  be  more  obvious 
at  some  level  of  job  analysis  finer  than  the  whole  job. 
For  Two-hand  Tracking,  conceptually  appropriate  tasks 
include 

-  Visually  guided  gunnery,  which  is  critical  in 
Infantry,  Tank  Crew,  some  Air  Defense  jobs, 

-  Driving,  which  is  a  separate  job,  and  is  also 
involved  in  Combat  Engineering,  Mechanized 
Infantry,  Tank  Crew 

Infantry  and  Tank  Crew  are  large  fill,  combat  critical 
MOS  which,  because  of  the  lack  of  similar  civilian  jobs, 
are  hard  to  fill.  Army  spends  large  sums  recruiting 
to  fill  those  specialties. 

4 .  Equating 

A.  Problems  in  developing  equivalent  forms  (e.q..  How  large 
is  the  potential  item  pool?  Would  pretesting  and 
vetting  of  new  items  be  needed?  Could  item  devel  be 
automated?  Equivalent  forms  would  be  easy  to  develop, 
either  from  the  limitless  population  of  potential  new 
items  or  from  scrambling  existing  items.  Item  develop- 
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ment  could  be  done  by  algorithm.  It  is  an  open  question 
whether  there  are  enough  items  in  existence  now  to  allow 
anticipating  how  new  items  would  behave. 

R.  Types  of  samples  needed/acceotable  for  initial  equating? 
Testing  conditions  needed?  The  need  for  enough 
minorities  and  low  AFQT  Examinees  would  drive  sample 
size. 

r.  Snharoup  dependence  of  ecruatinas;  To  be  determined  when 
equating  analyses  are  done. 

D.  Practicality  of  pencil-and-paoer  versions:  The  primary 
outcome  measure  is  sampled  every  hundredth  of  a  second, 
so  it  would  be  hard  to  replicate  on  paper. 

5.  Feasibility  for  operational  S/C 

A.  Min. /Max,  admin  times.  Times  for  instrux/test  for  low 
and  high  AFQT  examinees.  Mean  total  time  for 
instructions  plus  items  was  6m  01s  (s.d.  34s)  for  9,200+ 
incumbents  (p.  104).  Times  by  AFQT  TBD  under  the 
contract  that  follows  up  Project  A. 

B.  Susceptibility  to 

Cheating .  compromise ;  Probably  low 

Irrelevant  response  strategies:  An  examinee  could  try  to 
keep  the  crosshair  near  the  target  instead  of  following 
the  instruction  to  keep  it  on  the  target.  This  other 
strategy  is  not  "irrelevant”  since  it  would  involve 
tracking.  Whether  it  would  produce  results  that  are 
different  from  the  instructed  strategy  is  an  empirical 
question. 

Coaching:  There  may  be  coachable  but  relevant  response 
strategies  (things  like  "Keep  your  eye  on  the  ball"). 

Practice:  A  gain  of  .24  s.d.  was  found  in  480  incumbents 

c.  Floor /ceiling  effects?  Performance  of  low/hiqh  AFQTs: 
Mean  distance  off  target  has  a  large  maximum  score. 

Scores  by  AFQT  TBD  in  the  follow-on  to  Project  A. 

D.  Examinees*  reactions:  motivation,  enjoyment,  boredom: 
This  test  is  exciting  and  difficult. 

E.  Limitations  of  CAT-ASVAB  hardware  for  delivering  the 
test.  Additional  hardware  reouired:  Double  digital  to 
analog  game  board,  clock,  and  response  pedestal  with  two 
sliding  controls. 
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F.  Need  for  special  eopt  with  special  maintenance?  The 

byttons  and  controls  on  the  response  pedestal  sometimes 
need  repair  or  replacement. 
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The  case  for  Number  Memory  to  be  in  the  ECAT  Validation 

Description!  Number  Memory  is  a  28  item  mental  arithmetic  test 
from  the  computerized  battery  of  Project  A.  The 
examinees  are  shown  a  number,  then  at  their  own 
pace,  bring  on  a  new  screen  with  directions  for  an 
arithmetic  operation  to  be  performed  using  another 
number  (e.g..  Add  5).  After  a  series  of  such 
operations  and  numbers,  examinee  presses  a  button 
to  indicate  if  a  probe  number  is  the  answer  to  the 
series  of  operations.  Below  are  three  successive 
screens  printed  together. 


Start  with  in 
Divide  by  7 


1.  Theoretical  development  of  construct  and  measure (s) 


A.  Definition  of  construct  (process [es] ) : 

Cognitive  abilities,  quantitative:  ability  to  perform 
the  four  basic  arithmetic  operations  quickly  and 
accurately.  The  format  of  this  task  requires  the 
examinee  to  use  working  memory  as  well. 

How  is  this  construct  related  to  other  constructs/ 
factors? 

B.  Taxonomy/types/categories  of  items:  Items  vary  in  the 
number  of  steps  presented  before  a  solution  is  required 
(2,  3,  or  4) .  The  four  arithmetic  operations  were 
presented  equally  often. 

How  the  test  is  scored/scaled:  A  variety  of  speed  and 
accuracy  measures  are  available,  both  for  whole  problems 
and  for  each  of  the  four  arithmetic  operations.  The 
measure  which  is  used  for  analyses  below  is  the  pooled 
mean  operation  time.  Pooling  is  within  arithmetic 
operations  across  problems,  then  averaging  is  across 
operations. 

How  are  item  type  and  difficulty  jointly  sampled:  In 
a  balanced  fashion. 

C.  Seeming  differences  across  subgroups  in  experiences 
relevant  to  any  type  of  item  or  in  offensiveness  of 
content  (e.g.,  experience  with  computers):  There  is 
no  content,  except  arithmetic.  Scores  on  this  test 
did  not  significantly  correlate  with  self  reports  of 
video  game  experience  in  256  incumbents  in  the 
Project  A  Field  Test. 

D.  Basis  for  expecting  incremental  validity  in  specific 
jobs?  The  hypothesis  that  administering  a  test  like  NO 
by  computer  could  improve  its  statistical  characteristics 
and  validity.  In  the  expert  judgment  task  of  Project  A, 
Number  Memory  had  an  average  mean  expected  validity  of 
.40.  Technical  and  Clerical  MOS  were  expected  to  benefit 
the  most  from  this  ability. 

2.  Precision  of  measurement 

A.  Test's  consistency  across 

Different  forms:  One  form  exists  now 

Testing  occasions:  .73  for  mean  pooled  decision  time 
in  473  cases 

Levels  of  total  test  score  (score  conditional  SEM) : 

To  be  determined  in  the  contract  following  up  Project  A 
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B.  For  power  or  speeded  test;  internal  consistency;  Odd- 
even  reliability  of  .93  for  pooled  mean  decision 

time  samples  of  at  least  7,700  incumbents. 

C.  Variation  across  subgroups  in  score-conditional  SEM: 

To  be  determined  in  the  contract  following  up  Project  A 

D.  If  score  is  model-based,  what*s  the  model -based 
precision?  N.  A. 

E.  Sxibgroup  differences  in  item  functioning; 

Fairness  analyses  will  be  carried  out  under  the  contract 
that  follows  up  Project  A.  Means  and  s.d.'s  of  the  mean 
pooled  decision  time  by  group  in  the  Concurrent 
Validation  were  as  follows; 


Group 

n 

X 

s.d. 

Whites 

5,930 

222.96 

75.21 

Blacks 

2,485 

252.78 

83.34 

Hispanics 

324 

254.95 

80.48 

Other 

350 

245.36 

94.47 

Females 

8,210 

232.64 

80.45 

Males 

879 

237.60 

72.62 

.  Validity 

A.  Construct  validity;  correlations  with  other  tests  of 
similar  things.  Intercorr.  with  ASVAB;  In  factor 
analyses  with  ASVAB,  the  latency  measures  of  Number 
Memory  loaded  on  a  quantitative  factor  that  included  AR 
and  MK  (p.  81) .  In  other  factor  analyses  (pp.  82-5) , 
the  company  it  kept  varied  across  a  mixture  of  quanti¬ 
tative  and  speed.  Its  lowest  correlations  with  pre¬ 
enlistment  ASVAB  subtests  were  with  MC  and  El  (p.  87). 

B.  Uniqueness  re  ASVAB.  How  assessed?  In  terms  of  internal 
consistency  and  retest  reliability,  uniqueness  relative 
to  pre-enlistment  ASVAB  was  .66  and  .48  for  the  pooled 
mean  decision  time  measure. 

C.  Incremental  validity  re  ASVAB?  Improvement  in  SD  of  the 
predicted  criterion?  How  many  other  tests  were 
considered  when  this  one  was  validated?  Incremental 
validities  of  all  computerized  and  spatial  predictors 
from  Project  A  were  examined  in  GLM  analyses  by  Silva 
(1989,  unpublished).  The  sample  was  311  trainees  who 
had  taken  ASVAB  as  applicants,  the  Project  A  tests 
during  their  first  three  days  of  Service  (1986/87) ,  and 
then  performance  tests  on  their  anti-tank  gunnery  skills 
about  twelve  weeks  later.  The  criterion  measure  used 
here  was  accuracy  on  the  fifth  event  (event  =  set  of  10 
trials) ,  which  was  the  first  event  on  which  a  gunner 
could  attain  qualification.  Correlations  of  all  Project 


67 


A  tests,  AFQT,  and  ASVAB's  Combat  Composite  with  the 
criterion  were  first  examined.  Variance  in  the 
criterion  accounted  for  by  two  sets  of  tests  -  all 
Project  A  and  all  ASVAB  -  was  computed  separately  and 

then  same  analysis  was  run  on  the  Project  A  tests 
with  the  criterion  variance  associated  with  ASVAB 
removed.  Finally,  the  Project  A  tests  were  entered  into 
a  stepwise  linear  regression  on  the  residual  (unrelated 
to  ASVAB)  variance. 

Number  Memory  correlated  .069  (p  =  .195)  with  the 
criterion,  compared  with  .179  for  the  Combat  Composite 
(CS  +  AR  +  MC  +  AS)  and  .145  with  AFQT.  In  the  analyses 
with  all  ASVAB  and/or  other  Project  A  tests  entered  that 
are  summarized  on  p.  100,  it  did  not  contribute  signifi¬ 
cant  unicjue  variance.  However,  in  an  earlier  stepwise 
regression,  before  the  specification  of  the  predictor 
scores  had  been  cleaned  up.  Number  Memory  entered 
significantly  (second  variable  to  enter,  adding  of 
.0116  (p  =  .0491)).  The  difference  between  the  earlier 
analysis  and  the  one  on  p.  100  was  this:  the  score  for 
Perceptual  Speed  and  Accuracy  was  changed  from  percent 
correct  to  latency,  and  Number  Memory  was  changed  from 
mean  trimmed  latency  of  final  response  to  mean  latency  of 
the  component  responses  to  the  problems.  The  latter 
score  for  Number  Memory  seems  preferable  on  the  grounds 
that  it  captures  more  information,  and  the  small  sample 
size  invites  an  explanation  of  sampling  error  for  any 
unreplicated  finding.  Although  its  mean  final  response 
time  entered  significantly  in  the  earlier  stepwise 
analysis,  the  simple  correlation  of  the  non-preferred 
score  with  the  criterion  was  .0153  (n.s.).  In  the  two 
stepwise  regressions  where  Number  Memory  behaved 
differently,  the  set  of  other  predictors  that  entered 
significantly  remained  the  same. 

D.  Siibgroup  differences  in  incremental  validity  or  in  std 
err  of  prediction  of  the  criterion?  To  be  determined 
under  the  contract  that  follows  up  Project  A. 

E.  Job  families  and  patterns  of  validities  for  them. 
Similarity  of  job  families  in  the  other  Services. 

Expected  utility  in  terms  of  nvimbers,  attrition, 
training  costs,  attractiveness  to  applicants: 

Number  memory  should  have  utility  in  general,  across 
all  MOS,  as  the  quantitative,  speed,  and  accuracy 
attributes  that  it  is  related  to  are  generally  useful. 

4 .  Equating 

A.  Problems  in  developing  equivalent  forms  (e.g..  How  large 
is  the  potential  item  pool?  Would  pretesting  and  vetting 
of  new  items  be  needed?  Could  item  devel  be  automated?) 
The  potential  item  pool  is  very  large.  Item  development 
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probably  could  be  automated. 

B.  Types  of  seunples  needed/accept2d>le  for  initial  equating? 
Testing  conditions  needed?  Numbers  in  the  subgroups 
would  be  the  limiting  condition.  No  unusual  testing 
conditions. 

C.  Subgroup  dependence  of  equatings:  To  be  determined  when 
we  reach  the  stage  of  equating. 

D.  Practicality  of  pencil-and-paper  versions: 

Computerization  makes  the  variety  of  response  time 
measures  possible. 

5.  Feasibility  for  operational  S/C 

A.  Min. /Max.  admin  times.  Times  for  instrux/test  for  low 
and  high  AFQT  examinees:  The  mean  and  s.d.  of  total  test 
time  (instructions  +  items)  were  9m  32s  and  2m  26s  in 
9,000+  incumbents . 

B.  Susceptibility  to 

Cheating,  compromise,  irrelevant  response  strategies, 
coaching:  Judged  to  be  low 

Practice:  In  473  incumbents,  the  gain  in  mean  pooled 
decision  time  was  .44  s.d.  on  retesting.  Mean  decision 
time  showed  a  gain  of  .26. 

C.  Floor/ceiling  effects?  Performance  of  low/high  AFQTs 
on  the  test?  The  maximum  latency  is  unlimited.  How 
close  to  a  minimum  the  mean  latency  of  2.0  secs  on 
retesting  is  remains  to  be  seen.  Performance  by  AFQT 
level  remains  to  be  determined  in  the  contract  following 
up  Project  A. 

D.  Examinees'  reactions:  motivation,  enjoyment,  boredom: 
Nothing  noteworthy. 

E.  Limitations  of  CAT-ASVAB  hardware  for  delivering  the 
test.  Additional  hardware  required:  None 

F.  Need  for  special  eqpt  with  special  maintenance?  None 
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The  case  for  the  Short-term  Memory  Test  to  be  in 
the  ECAT  Validation 

Description:  Short-term  Memory  is  a  36-item  test  from  the 

computerized  battery  of  Project  A  that  involves 
judging  whether  a  single  symbol  on  the  video 
screen  was  in  an  array  of  symbols  (English  letters, 
Greek  letters,  printers'  symbols,  etc.)  that  just 
disappeared.  In  order  for  the  stimuli  to 
be  presented,  the  examinee's  hands  must  be  in  a 
standard  position  on  the  reponse  device.  Practice 
items  and  self-paced  instructions  are  given  on  the 
screen.  A  sample  screen  is  shown  below. 
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1.  Theoretical  development  of  constxruct  and  measure  (s) 


A.  Definition  of  construct  (process[es]) : 

Speed  and  accuracy  with  which  one  searches,  recalls,  and 
judges  information  in  short-term  memory  for  visually 
presented  material . 

How  is  this  construct  related  to  other  constructs/ 
factors?  Short-term  memory  underlies  perceptual  speed 
and  accuracy.  The  type  of  memory  measured  by  this  test 
may  not  be  as  deep  as  working  memory 

B.  Taxonomy/types/categories  of  items:  Items  vary  in  the 
number  of  symbols  in  the  stimulus  set  (1,  3,  or  5) , 
interval  between  offset  of  the  stimulus  set  and  onset 
of  the  probe  (.5  or  2.5  sec),  type  of  symbols  used 
(letters  vs.  relatively  unfamiliar),  and  correct 
answer  (''in”  [the  stimulus  set]  vs  "out”) . 

How  the  test  is  scored/scaled?  Proportion  correct  and 
mean  of  trimmed  (of  extreme  values)  decision  time  are 
the  primary  outcome  measures. 

How  are  item  type  and  difficulty  jointly  sampled? 

The  design  parameters  are  crossed. 

C.  Seeming  differences  across  subgroups  in  experiences 
relevant  to  any  type  of  item  or  in  offensiveness  of 
content  (e.g.,  experience  with  computers):  In  256 
incumbents  in  the  Project  A  Field  Test,  there  was  a 
significant  correlation  between  experience  with  video 
games  and  percent  correct  (r  =  .13)  and  a  non¬ 
significant  correlation  with  the  latency  outcome. 

D.  Basis  for  expecting  incremental  validity  in  specific  jobs 
In  the  judgment  of  Project  A's  expert  panel  (Wing, 
Peterson,  and  Hoffman,  1984),  visual  memory  will  have  a 
median  correlation  of  .21  with  14  tasks  of  inspecting, 
troubleshooting,  and  repairing  various  kinds  of  systems. 
For  controlling  air  traffic  and  for  tasks  making  use  of 
maps,  estimated  validities  are  in  the  30s. 

2.  Precision  of  measurement 

A.  Test's  consistency  across 

Different  forms:  One  form  of  the  test  exists  now. 

Testing  occasions:  In  474  incuinbents ,  retest  reliability 
was  .41  and  .66  for  the  accuracy  and  latency  measures. 

Levels  of  total  test  score  (score  conditional  SEM) : 

To  be  determined  under  the  contract  that  follows  up 
Project  A. 
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B.  For  power  or  speeded  ^est::  internal  consistency: 

.96  and  .60  odd-even  reliabilities  for  the  latency  and 
percent  correct  measures 

C.  Variation  across  subgroups  in  score-conditional  SEM: 

To  be  determined  under  the  contract  that  follows  up 
Project  A 

D.  If  score  is  model— based,  what*s  the  model— based 

precision:  N.A. 

E.  Subgroup  differences  in  item  functioning: 

Fairness  analyses  will  be  carried  out  under  the  contract 
that  follows  up  Project  A.  Means  and  s.d. 's  of  the 
latency  measure  by  group  in  the  Concurrent  Validation 
were  as  follows: 


Group 

n 

X 

S.d. 

Whites 

5,976 

86.94 

23.23 

Blacks 

2,483 

88.93 

25.16 

Hispanic 

330 

93.25 

28.03 

Other 

348 

87.39 

24.31 

Females 

886 

87.65 

22.09 

Males 

8,251 

87.73 

24.24 

3.  Validity 

A.  Construct  validity:  correlations  with  other  tests  of 
similar  things.  Intercorr.  with  ASVAB:  In  a  factor 
analysis  with  pre-enlistment  ASVAB  and  all  of  the 
Project  A  computerized  and  spatial  tests  (p.  81) ,  Short¬ 
term  Memory ' s  proportion  correct  loaded  . 3  on  a  factor 
named  Complex  Speed  and  Accuracy,  along  with  the 
accuracy  and  latency  scores  for  Target  Identification  and 
Perceptual  Speed  and  Accuracy.  Its  latency  score, 
however,  loaded  .37  with  a  factor  otherwise  formed  of 
scores  from  the  Simple  and  Choice  Reaction  Time  tests. 
These  two  scores  tend  to  split  apart  in  other  factor 
analyses  of  subsets  of  those  same  tests  (pp.  82-5) . 

B.  Uniqueness  re  ASVAB.  How  assessed?  In  terms  of  odd- 
even  reliability,  uniqueness  of  the  accuracy  and  latency 
outcomes  are  .55  and  .93,  in  order.  In  terms  of  retest 
reliability,  they  are  .36  and  .63,  also  in  order. 

C.  Incremental  validity  re  ASVAB?  Improvement  in  SD  of  the 
predicted  criterion?  How  many  other  tests  were 
considered  when  this  one  was  validated?  Incremental 
validities  of  all  computerized  and  spatial  predictors 
from  Project  A  were  examined  in  GLM  analyses  by  Silva 
(1989,  unpublished).  The  sample  was  311  trainees  who 
had  taken  ASVAB  as  applicants,  the  Project  A  tests 
during  their  first  three  days  of  Service  (1986/87) ,  and 
then  performance  tests  on  their  anti-tank  gunnery  skills 
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about  twelve  weeks  later.  The  criterion  measure  used 
here  was  accuracy  on  the  fifth  event  (event  =  set  of  10 
trials) ,  which  was  the  first  event  on  which  a  gunner 
could  attain  qualification.  Correlations  of  all  Project 
A  tests,  AFQT,  and  ASVAB's  Combat  Composite  with  the 
criterion  were  first  examined.  Variance  in  the 
criterion  accounted  for  by  two  sets  of  tests  ”  all 
Project  A  and  all  ASVAB  -  was  computed  separately  and 
then  same  analysis  was  run  on  the  Project  A  tests 
with  the  criterion  variance  associated  with  ASVAB 
removed.  Finally,  the  Project  A  tests  were  entered  into 
a  stepwise  linear  regression  on  the  residual  (unrelated 
to  ASVAB)  variance.  The  results  are  shown  on  p.  100. 

Short-term  Memory  correlated  .050  (n.s.)  with  the 
criterion,  compared  with  .179  for  the  Combat  Composite 
(CS  +  AR  +  MC  +  AS)  and  .145  with  AFQT.  In  various  GLM 
analyses  with  all  ASVAB  and  other  Project  A  tests.  Short¬ 
term  Memory  had  significant  unique  variance  (p  *  .024). 

In  the  stepwise  regression,  it  added  .0095  to  the  R^, 
entering  second. 

D.  Subgroup  differences  in  incremental  validity  or  in  std 
err  of  prediction  of  the  criterion?  To  be  determined 
under  the  contract  that  follows  up  Project  A 

E.  Job  families  and  patterns  of  validities  for  them. 
Similarity  of  job  families  in  the  other  Services. 

Expected  utility  in  terms  of  numbers,  attrition,  training 
costs,  attractiveness  to  applicants:  It  bears  repeating 
that  we  will  do  better  looking  for  gains  at  some  level 

of  job  analysis  finer  than  the  whole  job.  The  tasks 
for  which  Project  A's  expert  panel  expected  Short-term 
Memory  to  have  useful  validities  have  their  counterparts 
in  jobs  across  the  Services. 

4 .  Equating 

A.  Problems  in  developing  equivalent  fozms  (e.g..  How  large 
is  the  potential  item  pool?  Would  pretesting  and  vetting 
of  new  items  be  needed?  Could  item  devel  be  automated? 
The  design  parameters  for  the  items  being  as  objective 

as  they  are,  automation  should  be  possible.  In  order  to 
keep  the  unfamiliar  symbolic  stimuli  from  becoming 
familiar  to  recruiters  and  applicants,  we  may  want  to 
enlarge  the  pool  of  those  symbols. 

B.  Types  of  samples  needed/acceptable  for  initial  equating? 
Testing  conditions  needed?  No  unusual  testing 
conditions.  Samples  with  sufficient  n's  in  the  smaller 
subgroups 

C.  Subgroup  dependence  of  eqpiatings:  To  be  determined  when 
we  get  to  the  stage  of  equating. 
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D.  Practicality  of  pencil-and-paper  versions; 

Computerization  makes  this  kind  of  test  possible 

5.  Feasibility  for  operational  S/C 

A.  Min. /Max.  admin  times.  Times  for  instrux/test  for  low 
and  high  AFQT  examinees:  For  the  9,269  cases  in  the 
Concurrent  Validation,  the  mean  and  s.d.  of  time  for  the 
test  +  instructions  were  7m  35s  and  Im  27s  (p.  104) . 

B.  Susceptibility  to 

Cheating,  compromise:  As  long  as  the  item  pool  is 
large  enough,  probably  low 

Irrelevant  response  strategies,  coaching:  None  that 
we  are  aware  of 

Practice;  Gains  in  accuracy  and  latency  were  .17  and  .15 
in  474  incumbents 

C.  Floor/ceiling  effects?  Performance  of  low/high  AFQTs 
on  the  test?  Performance  by  AFQT  level  will  be 
determined  under  the  contract  that  follows  up  Project  A. 
With  a  percent  correct  of  .88  (s.d.  .08)  on  retesting 
there  may  be  some  squeezing  of  scores  at  the  top. 

One  s.d.  below  the  retest  mean  is  about  .6  sec.  How 
much  that  could  decline  is  yet  to  be  determined. 

D.  Examinees*  reactions:  motivation,  enjoyment,  boredom: 
Nothing  noteworthy 

E.  Limitations  of  CAT-ASVAB  hardware  for  delivering  the 
test.  Additional  hardware  recjuired:  A  key  arrangement 
is  needed  to  assure  that  examinees  start  their  keypresses 
from  a  standard  hand  position. 

F.  Need  for  special  eqpt  with  special  maintenance? 

None 
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The  case  for  Target  Identification  to  be  in  the  ECAT  Validation 


Description: 


Target  Identification  is  a  48-item  test  from  the 
computerized  battery  of  Project  A  that  involves 
pressing  a  button  to  pick  from  three  outline 
figures  low  on  the  video  screen  the  one  that  is 
just  like  a  target  figure  at  the  top  of  the  screen. 
The  three  choices  may  be  rotated  from  the  position 
of  the  target  figure.  In  order  for  the  stimuli  to 
be  presented,  the  examinee's  hands  must  be  in  a 
standard  position  on  the  reponse  device.  Practice 
items  and  self-paced  instructions  are  given  on  the 
screen.  A  sample  screen  is  shown  below. 


luoi 
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1.  Theoretical  development  of  construct  and  measure (s) 


A.  Definition  of  construct  (process Tesl ) ; 

Domain:  Perceptual  speed  and  accuracy 

The  ability  to  make  rapid,  accurate  comparisons  between 
two  visual  stimuli.  Scanning  and  working  memory  are 
involved.  For  this  test,  which  varies  the  orientation 
of  the  target  stimulus,  mental  rotation  is  involved. 

Wow  is  this  construct  related  to  other  constructs/factors 
See  below  3. A.  Construct  Validation 

B.  Taxonomv/tvpes/cateaories  of  items; 

Four  stimulus  dimensions  were  varied;  type  of  target 
(e.g.,  tank,  fixed  wing  aircraft,  — ),  position  of 
correct  response  choice  (left,  middle,  right) ,  angle  of 
rotation  from  the  normal  upright  position,  and  match 
between  right-left  orientation  of  target  and  response 
choices. 

How  the  test  is  scored/scaled?  The  primary  scores 
(p.  )  are  Proportion  Correct  and  Mean  Decision  Time. 

Decision  Time  is  the  time  from  stimulus  onset  to  onset 
of  hand  motion. 

How  are  item  type  and  difficulty  jointly  sampled;  Two 
items  were  created  for  each  cell  of  a  table  that  crosses 
the  design  parameters.  The  parameters  all  have  a 
significant  effect  on  item  difficulty. 

C.  Seeming  differences  across  subgroups  in  experiences 
relevant  to  anv  tvoe  of  item  or  in  offensiveness  of 
content  fe.o..  experience  with  computers) 

In  256  incumbents  in  the  Project  A  field  test,  the 
accuracy  and  speed  scores  on  Target  Identification 
correlated  .05  -  .08  (n.s.)  with  self  reports  of 
experience  with  video  games.  Such  experience  may  be 
related  to  SES.  Offensiveness  seems  not  to  be  an 
issue,  as  the  stimuli  are  planes,  helicopters,  etc. 

D.  Basis  for  expecting  incremental  validity  in  specific  jobs 

-  Except  that  it  uses  diagrammatic  graphics,  it  bears 
little  resemblance  to  ASVAB  subtests; 

-  It  resembles  the  military  task  of  identifying  real 
targets  quickly  and  accurately; 

-  In  the  judgment  of  Project  A  experts  (Wing,  Peterson, 
and  Hoffman,  1984) ,  perceptual  speed  and  accuracy  are 
likely  to  predict  a  variety  of  aspects  of  enlisted 
performance,  including:  inspecting,  troubleshooting, 
and  repairing  various  types  of  systems;  sending/ 
receiving  radio  messages;  operating  keyboard  devices; 
detecting  and  identifying  targets;  firing  direct  fire 
weapons;  a  variety  of  clerical  tasks;  translating 
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ifferent  forms 


Split  half  reliabilities  were  .97  and  .62  for  the  time 
and  accuracy  scores. 

Variation  across  subgroups  in  score-conditional  SEM; 

To  be  determined  under  the  contract  that  follows  up 
Project  A. 

If  score  is  model-based,  the  model~based  precision;  N. A. 

Subgroup  differences  in  item  functioning; 

Fairness  analyses  will  be  carried  out  under  the  contract 
that  follows  up  Project  A.  Means  and  s.d.'s  of  the 
decision  time  measure  by  group  in  the  Concurrent 
Validation  were  as  follows: 


Group 

n 

X 

s.d. 

Whites 

5,939 

180.19 

55.05 

Blacks 

2,479 

222.45 

68.64 

Hispanics 

324 

213.08 

68.51 

Other 

351 

201.38 

71.61 

Females 

878 

223.33 

66.23 

Males 

8,215 

190.53 

61.98 

Validit^ 


Construct  validity:  correlations  with  other  tests  of 


similar  things.  Intercorr.  with  ASVAB: 

Mean  decision  time  loads  most  heavily  on  the  factor  of 
perceptual  speed/accuracy,  but  has  moderate  secondary 
loadings  with  spatial  and  psychomotor  tests  (pp  81-5) . 
Among  the  ASVAB  subtests  (p.  87) ,  the  decision  time 
score  correlates  highest  with  MC  (.33),  GS  (.30),  and  AS 
(.25) . 

Unigueness  re  ASVAB.  How  assessed?  .83  in  terms  of  odd 
even  reliability;  .64  relative  to  retest  reliability. 
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C.  Incremental  validity  re  ASVAB?  Improvement  in  SD  of  the 
predicted  criterion?  How  many  other  tests  were 
considered  when  this  one  was  validated?  Incremental 
validities  of  all  computerized  and  spatial  predictors 
from  Project  A  were  examined  in  GLM  analyses  by  Silva 
(1989,  unpublished).  The  sample  was  311  trainees  who 
had  taken  ASVAB  as  applicants,  the  Project  A  tests 
during  their  first  three  days  of  Service,  and  then 
performance  tests  of  their  anti-tank  gunnery  skills 
about  twelve  weeks  later.  The  criterion  measure  was  an 
accuracy  score  on  the  fifth  event  (i.e.,  set  of 

10  trials) ,  which  was  the  first  event  on  which  a  gunner 
could  attain  gualification.  Correlations  of  all  Project 
A  tests,  AFQT,  and  ASVAB 's  Combat  Composite  with  the 
criterion  were  first  examined.  Variance  in  the  criterion 
accounted  for  by  two  sets  of  tests  -  all  Project  A  and 
all  ASVAB  -  was  computed  separately  and  then  same 
analysis  was  run  on  the  Project  A  tests  with  the 
criterion  variance  associated  with  ASVAB  removed. 

Finally,  the  Project  A  tests  were  entered  into  a  stepwise 
linear  regression  on  the  residual  (unrelated  to  ASVAB) 
variance.  Results  are  shown  on  p.  100. 

The  decision  time  score  for  Target  Identification 
correlated  .174  with  the  criterion,  compared  with  .179 
for  the  Combat  Composite  (CS  +  AR  +  MC  +  AS)  and  .145 
with  AFQT.  The  multiple  R^  for  all  ASVAB  subtests  was 
.041,  for  all  Project  A  was  .166,  and  for  both  sets 
together  .177.  Project  A  tests  accounted  for  .126  of  the 
residual  variance. 

The  unique  variance  accounted  for  by  Target 
Identification  had  a  p  between  .05  and  .10  when  all  ASVAB 
and  other  Project  A  tests  were  included  in  the  multiple 
regression  and  in  analyses  of  Project  A  tests  alone.  In 
a  stepwise  analysis  of  Project  A  tests  predicting  the 
non-ASVAB  related  criterion  variance.  Target  ID  is  the 
third  of  five  tests  to  enter  (p  =  .0403),  adding  .0125  to 
the  r2. 

D.  Subgroup  differences  in  incremental  validity  or  in  std 
err  of  prediction  of  the  criterion?  To  be  determined 
under  the  contract  that  follows  up  Project  A. 

E.  Job  families  and  patterns  of  validities  for  them. 
Similarity  of  iob  families  in  the  other  Services. 

Expected  utility  in  terms  of  numbers,  attrition,  training 
costs,  attractiveness  to  applicants:  See  above  pp.  38-9. 
Target  identification  is  itself  a  job  task  in  air 
defense,  tank  and  anti-tank  gunnery,  and  fire  support. 

As  an  indicator  of  perceptual  speed  and  accuracy,  this 
test  should  predict  performance  in  radio  operating, 
typing,  translating,  decoding,  and  controlling  air 
traffic.  These  are  all  pan-Service  activities. 
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4 .  Ecfuatina 

A.  Problems  in  developing  equivalent  forms  (e.a..  How  large 
is  the  potential  item  pool?  Would  pretesting  and  vetting 
of  new  items  be  needed?  Could  item  devel  be  automated?) 
The  potential  item  pool  is  very  large  when  you  multiply 
types  of  targets  (truck,  helicopter,  tank,  etc.)  by 
orientation  by  composition  of  the  set  of  response  choices 
by  position  of  the  correct  choice.  New  items  would  need 
to  be  pretested  at  least  for  debugging.  Whether  the 
existing  item  pool  is  large  enough  to  permit  estimation 
of  item  parameters  for  all  other  potential  items  needs 
to  be  determined. 

B.  Types  of  samples  needed/acceptable  for  initial  ecruatinq? 
Testing  conditions  needed?  Adequate  numbers  of 
minorities  and  low  AFQT  examinees  will  be  hardest  to  get. 
Existing  data  from  ASVAB  R&D  should  show  whether  recruits 
respond  like  applicants. 

c.  Rnhrr-rniip  dependence  of  eouatinas;  To  be  determined  when 
equating  becomes  an  issue  for  this  test. 

D.  Practicality  of  pencil-and-paoer  versions 

In  principle  this  test  could  be  given  on  paper,  but  that 
would  result  in  loss  of  the  best  response  measure:  mean 
decision  time. 

5.  Feasibility  for  operational  S/C: 

A.  Min. /Max,  admin  times.  Times  for  instrux/test  for  low 
and  high  AFQT  examinees:  Total  test  time  averages  4m 
33s,  s.d.  Im  15s.  Correlation  of  time  to  read  the 
directions  with  AFQT  is  around  .30. 

B.  Susceptibility  to: 

Cheating,  compromise:  Given  the  large  number  of 
potential  items,  the  hazard  seems  low. 

Irrelevant  response  strategies:  Validity  could  be  under¬ 
mined  by  examinees'  sacrificing  accuracy  for  speed.  To 
counter  that  tactic,  a  composite  predictor  could  be 
developed  from  latency  adjusted  for  accuracy. 

Coaching:  Doubtful 

Practice:  The  gain  on  retesting  473  incumbents  was  .32 
s.d.  The  relative  contribution  to  that  gain  of 
familiarity  with  items  and  test-wiseness  needs  to  be 
examined . 

C.  Floor/ceiling  effects?  Performance  of  low/high  AFQTs 
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on  -the  test?  Maximum  mean  decision  time  is  great  enough 
not  to  be  limited.  On  retesting,  the  mean  time  was 
about  1.3  sec  longer  than  mean  decision  time  for  the 
retest  of  choice  reaction  time. 

D.  Examinees*  reactions;  motivation,  enjoyment,  boredom: 
Nothing  noteworthy 

E.  Limitations  of  CAT-ASVAB  hardware  for  delivering  the _ 

test.  Additional  hardware  recmired;  This  test  uses  the 
four  home  buttons  on  the  reponse  pedestal,  to  assure  a 
standard  hand  position  for  responding,  plus  three 
response  buttons. 

F.  Need  for  special  ecrot  with  special  maintenance? 

No  other 
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Source:  unpublished  IPR  materials  by  N.  B.  Peterson, 
et  al.,  PDRI,  1986 
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Vrincfp.  #«etor  wwlytlt,  cowwunallty  Mtlaat*  •  aquirad  lultlpU  eorrtlatlon,  varlan  rotation. 
Mott:  N  >  7062 


Source:  unpublished  IPR  materials  by  N.  B.  Peterson, 
et  al.,  PDRI,  1986 
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Prfnclpl^^factor  anatysft,  comnunalfty  estfMte  *  tquared  iultfple  correlation,  vartiMx  rotation, 


Source: 


unpublished  IPR  materials  by  N.  B.  Peterson, 
et  al.,  PDRI,  1986 
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Source:  Smith  &  Graham,  1987 
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^11  .07  -vQO  -%04  -vM 


Concurrent  Validity  Data  Analysis 
Factor  Analysis^  of  Computer>Adm1n1stered  Measures 


Source:  unpublished  IPR  materials  by  N.  B.  Peterson, 
et  al.,  PDRI,  1986 
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frtnelptt  factor  anotyafa,  caamunoltty  catfinota  ■  aquarcd  auttipla  eerretatlon,  varlawR  rotation. 
Notot  N  •  8521 


Concurrent  Validity  Data  Analysis 
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Source;  unpublished  IPR  materials  by  N.  B.  Peterson, 
et  al.,  PDRI,  1986 


Concurrent  Validity  Data  Analysis 
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note:  M  range  ■  753A-78A3.  Distance  and  time  score  correlations  have  been  reflected 


Factor  Loadings  for  Paper  and  Pencil  Cognitive  Tests-Score  Correct 
Two>Factor  Solution  with  Orthogonal  Varlmax  Rotation 
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Source:  unpublished  IPR  materials  by  N.  B.  Peterson 
et  al.,  PDRI,  1986 


Factor  Loading^  Matrix  for  Cognitive  Test  ScQgei 
Defining  Alternative  Composite 


Source : 


unpublished  IPR  materials  by  N.  B.  Peterson, 
et  al.,  PDRI,  1986 
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Concurrent  Validity  Data  Analysis 

Correlations  of  Thirteen  Computer -Adalnistered  Psychoaiotor  Test  Scores 
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Source:  unpublished  IPR  naterlals  by  N.  B.  Peterson, 
et  al.,  PDRI,  1986 
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Nit/Miss  Proportion  scores'  correlations  with  distance  measures  hove  been  reflected. 
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Source:  unpublished  IPR  materials  by  N.  B.  Peterson, 
et  al.,  PDRI,  1986 
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^imes  are  given  in  hundredths  of  seconds.  Logs  are  natural  logs. 
2m»460-479  for  test- retest  correlations.  Retest  interval  was  4  to  6  weeks 
^Coefficient  Alpha  reliability  estimates. 


Concurrent  Validity  Data  Analysis 

Cognitive  Paper-and-Pencil  Measures:  ^  ♦ 

Odd-Even  Reliability  Coefficients,  Squared  Multiple  Regression  Coefficients 

(Aqainst  All  ASVAB  Subtests) 


Test 

li 

Odd-Even 

Reliability 

Squared  . 
Multiole  R^ 

Uniqueness 

Estimate 

Assembling  Objects 

9343 

.91 

.26 

.65 

Object  Rotation 

9345 

.98 

.18 

.80 

Maze  Test 

9344 

.96 

.22 

.74 

Orientation  Test 

9341 

.89 

.29 

.60 

Map  Test 

9343 

.90 

.44 

.46 

Reasoning  Test 

9332 

.87 

.34 

.53 

Source:  unpublished  IPR  materials  by  N.  B.  Peterson, 

et  al.,  PDRI,  1986 


^Ns  for  these  calculations  are  7329. 
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Source:  unpublished  IPR  materials  by  N.  B.  Peterson, 
et  al.,  PDRI,  1986 
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7516  for  these  calculations. 

lueness  -  Reliability  minus  Squared  Multiple 


Source : 


unpublished  IPR  materials  by  N.  B.  Peterson, 
et  al.,  PDRI,  1986 


Concurrent  Validity  Data  Analysis 


Odd-Even  Reliability  Coefficients,  Squared  Multiple  ^ 

Coefficients  All  ASVAB  Subtests),  and  Uniqueness  Estimates 

for  Sixteen  Computer  Administered  Lognitive/Perceptual  Test  Scores 


Score  M 

Simple  Reaction  Time; 

Trimmed  Decision  Time  Mean  9254 
Proportion  Correct  9254 

^;hnice  Reaction  Time; 

Trimmed  Decision  Time  Mean  9268 

Proportion  Correct  9268 

Choice  (DT)  -  Simple  (OT)  9249 

Short  Term  Memory; 

Trimmed  Decision  Time  Mean  9148 

Proportion  Correct  9148 

Perceptual  Speed  &  Accuracy: 

Trimmed  Decision  Time  Mean  9243 

Proportion  Correct  9243 

Target  Identification: 

Trimmed  Decision  Time  Mean  9104 

Proportion  Correct  9104 

Number  Memory; 

Final  Response  Time  Mean  9099 

Input  Time  Mean  9099 

Pooled  Operation  Time  Mean  9099 

Proportion  Correct  9099 

SRT-CRT-STM-PSA-TID: 

Pooled  Movement  Time  Mean  8962 


Odd-Even  Squared^  Uniqueness 


Reliability 

Multiole  R 

Estimate 

.88 

.01 

.87 

.46 

.02 

.44 

.97 

.04 

.93 

.57 

.02 

.55 

.85 

.01 

.84 

.96 

.03 

.93 

.60 

.05 

.55 

.94 

.02 

.92 

.65 

.04 

.61 

.97 

.14 

.83 

.62 

.03 

.59 

.88 

.21 

.67 

.10 

.85 

.9i2 

.25 

.66 

.59 

.20 

.39 

.74^ 

.03 

.71 

^Ns  for  these  calculations  range  from  7715  to  7860. 
^These  are  Coefficient  Alpha  reliabilities. 
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Concurrent  Validity  Data  Analysis 


Source:  unp:iblished  IPR  aaterials  by  N.  B.  Peterson, 
et  al.,  PDRI,  1986 


d 

CO 

o 

CO 

c 

o 

o 

CM 

Cl 

o 

M 

•H 

• 

• 

• 

• 

• 

• 

<0 

o 

o 

CO 

O 

n 

so 

cn 

CO 

OS 

so 

CO 

in 

o 

• 

• 

• 

• 

• 

• 

CO 

1^ 

^  ■ 

so 

in 

in 

M 

M 

n> 

SO 

fH 

pH 

t*  c 

o 

pH 

CM 

so 

<0 

• 

• 

• 

• 

• 

• 

•• 

0) 

fO 

CO 

n 

CO 

o 

to 

s 

M 

so 

H 

pH 

CM 

V 

u 

a 

to 

(0  (0 

^  O 

X  4J 

Q) 

•H  « 

•H  1 

O 

ro 

OS 

n 

00 

0 

o 

SO 

o 

in 

O 

C  (0 

CO 

• 

m 

• 

• 

• 

• 

0)  0) 

H 

SO 

OS 

SO 

in 

in 

04 

pH 

0) 

•d  u 

6 

C  0 

•H 

10 

C 

iH 

O 

in 

so 

■  10 

CO 

in 

c* 

pH 

u  to 

0) 

• 

• 

• 

• 

• 

• 

0)  0) 

£ 

fO 

•H 

so 

pH 

OS 

a  u 

fS 

SO 

pH 

pH 

pH 

(0  0 

04  O 

CO 

0) 

>  c 

•H  -H 

(0 

•H  O 

oa 

OS 

CM 

o 

Cl 

O 

c 

£ 

o 

OS 

O 

o 

o 

O 

in 

in 

in 

in 

in 

0 

u 

to 

O 

O 

44 

m 

C 

to 

0 

0) 

44 

O 

•H 

Eh 

to 

*P 

o 

10 

C 

Eh 

O' 

44 

0 

c 

O 

44 

•pH 

trs 

•H 

Pi 

to 

44 

44 

C 

pH 

0) 

to 

to 

•H 

A 

44 

44 

0) 

c 

1 

O 

C 

EH 

o 

0) 

0) 

o 

o 

to 

to 

N 

•H 

Oi 

10 

to 

43 

10 

(0 

o 

< 

O 

X 

o 

X 

X 

97 


®Gain  «  (Mj  -  M3^)/SDi 


n't,  Meant,  Standard  Oevlatlant,  and  Cain  Scores 
ror  Thirteen  CoatJUter-hdalnlatered  Saychawtor  Test  Scores 


Source:  unpublished  ZPR  materials  by  N.  B.  Peterson, 
et  al.,  PDRI,  1986 
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rent  Validity  Data  Analyila 
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TOW  data:  Predicting  Event  5*  with  14  Project  A  Predictors, 
ASVAB,  and  Combinations  Thereof 


,Source:  Jay  Silva,  API  (1989,  unpublished) 


Predictor 


r  with 
Event  S 


ASVAB 

AFQT  (uncorrected) 

CO  composite  (uncor) 
(CS+AR+AS+MC) 

All  sxibtests 

(optimal  combo, 
uncorrected)* 

Project  A 

14  Project  A  tests  below 
(optimal  combo  w*out 
removing  ASVAB 
related  variance) 

14  Project  A  tests 
(optimal  combo  to 
predict  ASVAB 
residualized  TOW) 

Computerized  tests 
Cannon  Shoot 
(Mean  abs  time  err) 
Short  Term  Memory 
(Mean  trimmed 
decision  time) 

Number  Memory 
(Mean  pooled 
response  time) 
Perceptual  Speed  and 
Accuracy  (Mean  trimmt 
decision  time) 

Target  Identification 
(Mean  trimmed 
decision  time) 

Target  Shoot 
(Mean  log[dist4l)) 
One-hand  Tracking 
(Mean  logtdist-fl]) 
Two-hand  Tracking 
(Mean  log[dist4i] ) 

Spatial  (4  correct) 
Assembling  Objects 
Map 
Mazes 

Object  Potation 
Orientation 
Reasoning 

ASVAB  combo  plus 
Project  A  combo 


.145 

•  0063 

353 

.180 

.0007 

353 

.203 

.1029 

353 

.407 

.0001 

314 

.354 

.0002 

311 

p  unic[ue 

stepwise 

res id  var 

R2  p/order 

.159 

.0029 

348 

.050 

.3533 

350 

.024 

.0095  .0757/2 

.069 

.1954 

349 

.894 

1 

.052 

.0392 

352 

.626 

.174 

.0012 

344 

.091 

.0125  .0403/3 

.112 

.0386 

340 

.824 

.206 

.0001 

352 

.842 

.294 

.0001 

353 

.003 

.0725  .0001/1 

.082 

.1305 

338 

.103 

.0088  .0844/4 

.161 

.0030 

338 

.694 

.108 

•  0482 

338 

.200 

.168 

.0018 

344 

.281 

.242 

•  0001 

340 

.041 

.0117  .0453/5 

.151 

•  0050 

344 

•  562 

.421 

.0001 

311 

d  in 

1987.  From 

1987 

to  1988, 

the 

** 


MC  was  the  only  stibtcst  accounting  for  significant  (p 
unigue  variance.  All  others  had  p  >  *4. 


3. 
.05) 
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TOW  data; 


Predicting  Event  3*  with  Four  Project  A  Predictors, 
ASVAB,  and  Combinations  Thereof 

Source;  Jay  Silva,  ARI  (1989,  unpublished) 


PforS  i  ctor  r  with  Event 

3  P - 

n  SEM 

AFQT  (uncorrected) 

9  ASVAB  subtests 
(unit  wtd  combo, 
no  corrections) 

One -hand  Tracking 
Two-hand  Tracking 

Mazes 

Orientation 

146 

191 

290 

332 

020 

162 

.0001 

.0001 

.0001 

.0001 

.754 

.0113 

1,747 

1,747 

245 

245 

245 

245 

9  ASVAB  subtests 
(optimal  combo, 
no  corrections) 

,266 

.0001 

1,747 

Four  Project  A  tests 
(optimal  combo  without 
regard  to  ASVAB) 

.374 

.0001 

245 

Contribution  to  model; 
One-hand  Tracking 
Two-hand  Tracking 
Mazes 

Orientation 

P 

for  unique  variance 
.54 
.005 
.041 
.035 

Four  Project  A  tests 
(optimal  combo  for 
predicting  residual 
after  ASVAB-related 
variance  is  removed) 

.308 

.0001 

245 

Contribution  to  model; 
One-hand  Tracking 
Two-hand  Tracking 
Mazes 

Orientation 

1 

1 

p  for  unique 
.43 
.029 
.010 
.817 

variance 

ASVAB  combo  plus 
Project  A  combo 

.421 

.0001 

245 

Not:e: 

Best  model  for  predicting  ASVAB  residualized  variance,  based 
on  highest  adjusted  R^,  best  c(p),  and  forward  entry 
stepwise  regression;  Two-hand  Tracking  +  Mazes. 

*Based  on  data  gathered  in  1988.  From  1987  to  1988,  the 
first  qualifying  table  was  moved  from  Event  5  to  Event  3. 
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UCOFT  data:  Predicting  the  speed/accuracy  Composite  With  Four 
Project  A  predictors,  ASVAB,  and  Combinations  Thereof 

Source:  Jay  Silva,  API  (1989,  unpublished) 


Predictor  ^  with  Speed/ 

Accuracy _ 

P 

n 

AFQT  (uncorrected) 

9  ASVAB  subtests 
(unit  wtd  combo, 
no  corrections) 
One-hand  Tracking 
Two-hand  Tracking 

Mazes 

Orientation 

284 

323 

484 

504 

308 

328 

.0001 

.0001 

.0001 

.0001 

.0001 

.0001 

498 

498 

394 

394 

326 

326 

9  ASVAB  subtests 
(optimal  combo, 
no  corrections) 

480 

.0001 

498 

Four  Project  A  tests 
(optimal  combo  without 
regard  to  ASVAB) 

.501 

.0001 

260 

Contribution  to  model: 
One-hand  Tracking 
Two-hand  Tracking 
Mazes 

Orientation 

p  for  unique  variance 
.41 
.004 
.015 
.024 

Four  Project  A  tests 
(optimal  combo  for 
predicting  residual 
after  ASVAB-related 
variance  is  removed) 

.312 

.0001 

259 

Contribution  to  model: 
One-hand  Tracking 
Two-hand  Tracking 
Mazes 

Orientation 

p  for  unique 
.28 
.121 
.025 
.17 

variance 

ASVAB  combo  plus 
Project  A  combo 

.565 

.0001 

259 

Note  • 

Best  model  for  predicting  ASVAB  residualized  variance,  based 
on  highest  adjusted  F? ,  best  c(p),  and  forward  entry 
stepwise  regression  includes  at  least  Two-hand  Tracking  and 
Mazes.  There  is  mixed  support  also  for  Orientation. 
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TabU  6.3 

eftrr«1it1on»  Cfluiouttr  T«it  Seorti  and  Pravioui  Emerltnce  With 

video  filmed  {H  ■  2S0)* 


Computer  Test 

Test  Score 

Correlation* 

Simple  Reaction  Time 

Mean  RT 

.12* 

Choice  Reaction  Time 

Mean  RT 

.15* 

Perceptual  Speed  S  Accuracy 

Nrcent  Correct 

-.01 

Mean  RT 

.01 

Slope 

•.03 

Intercept 

.06 

Target  Identification 

Nrcent  Correct 

.08 

Mean  RT 

.05 

Short'Term  Nernory 

Nrcent  Correct 

.13^ 

Mean  RT 

.08 

Slope 

-.16* 

Intercept 

.18* 

Number  Nernory 

Nrcent  Correct 

.08 

Mean  RT 

.00 

Cannon  Shoot 

Time  Error 

.18* 

Target  Tracking  1 

Mean  Leg  Distance 

.22* 

Target  Shoot 

Mean  Time  to  Fire 

.10 

Mean  Log  Distance 

.27* 

Target  Tracking  2 

Mean  Leg  Distance 

.16* 

*  Varlts  slightly  by  last. 

^  Corralatlons  of  .12  or  greater  are  statistically  signifl* 
cant  at  the  .05  level,  two>ta11ed  test  of  significance. 
Signs  of  correlations  have  been  reflected,  where  appro* 
priate,  so  that  greater  video  experience  shows  positive 
correlation  with  better  test  perforwance. 


Correlations  with  Video  e«iwe.Pl«v1no  gxaerlenen 

Table  6.3  shows  correlations  of  the  19  coaiputer>adw1n1stered  test 
scores  with  the  subject’s  previous  experience  playing  video  gaoes.  In  the 
caaputer>adw1n1 stared  tests,  the  question  was  asked:  *In  the  last  couple 
years,  how  nch  have  you  played  video  gaaes  on  arcade  sachlnes,  home  video 
games  or  home  computers?*  Subjects  selected  one  of  the  following  five 
answers:  •You  have  MEVER  played  video  games,*  *you  have  tried  a  few  games, 
but  have  generally  played  less  than  once  a  month,*  *Vou  have  played  several 
times  a  month,*  *yeu  have  played  at  least  once  or  twice  a  week,*  'you  have 
played  video  games  almost  every  day.*  These  answers  were  given  numeric 
values  from  1  to  S,  respectively.  The  mean  score  on  this  question  was 
2.99,  SO  ■  1.03  (N  •  256}  and  the  tast-retest  reliability  was  .71  (N  - 
113}  i 


Source:  Peterson,  1987 
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Concurrent  Validity  Data  Analysis 

Time  Used  to  Read  Instructions  and  Complete  Test  Items 
For  Computer-Administered  Tests 


pw>qraphics! 

tifTple  tgaction  Time; 

pho^ce  teaetion  Timer 

^hort  Terff  Myrcry; 

Target  Tracking  1: 


»»r<od 


Inatruction 
Ttst  Itw 
Total 

Instruction 
Test  Ittms 
Total 

Instruction 
Ttst  IttfBS 
Total 

Instruction 
Ttst  Ittms 
Total 

Instruction 
Ttst  Ittms 
Total 


Perceptual  Sneed  I  Accuracy;  Instruction 

Ttst  Ittms 
Total 

Instruction 
Ttst  Ittms 
Total 

Instruction 
Ttst  Ittms 
Total 


Target  Tracking  ?; 


iijnber  Memory; 


Cannon  Shoot: 


Target  Identification: 


Target  Shoot! 


Instruction 
Ttst  Ittns 
Total 

Instruction 
Ttst  Ittms 
total 


9273 

9273 


i/A 

i/A 

9271 

9272 
9272 


9269 

9269 


9256 

9257 


9266 

9266 


9254 

9253 


9256 

9256 


Instruction  9238 
Ttst  Items  9237 
Total 


0  mins  19  atcs 
A  mins  1  ate 
4  mins  20  atcs 

M/A 

M/A 


C  €0  itfta 
Imn  t^MCS 
1  acjcs 

a/A 

t/A 


9232 

9232 


9208 

9208 


Total  Time  for  tattary:  Total  Ttst  Time  9200 

Mote:  t*n9*  for  Totol  T«t  Tl*e  wM  ■Inotet  17 


2  mins 

20  atcs 

0  ai  ns 

58  atcf 

1  min 

0  atcs 

0  fi^s 

21  atca 

1  min 

38  atcs 

0  fira 

22  atca 

2  mins 

58  atcs 

0  st/it 

44'  sees 

2  mins 

61  atcs 

0  r.?a 

47  sera 

4  mins 

54  atcs 

0  arns 

44l  s«;o 

7  mins 

35  atcs 

1  tin 

27  atcs 

3  mins 

37  atcs 

0  ens 

VL  sees 

3  mins 

47  atcs 

0  mirs 

2  sees 

7  mins 

24  sees 

0  mirs 

5*  ltC£ 

1  min 

44  atcs 

0  mra. 

U  stet 

3  mins 

25  sees 

0  an*.s 

14  sues 

S  mins 

9  sees 

1  mro 

18  stes 

2  mins 

14  atcs 

0  tins 

n  secs 

3  mins 

47  atcs 

0  wArs 

!i  atx 

6  mins 

1  ate 

0  m'ts 

34  stca 

3  mins 

14  atcs 

1  mr\ 

tr  stca 

4  mins 

18  atcs 

1  mh 

15  stes 

9  mins 

32  atcs 

2  m?Tvr. 

24  secs 

3  mins 

8  atcs 

1  mir. 

3^  f«3S 

3  mins 

43  atcs 

0  mirr. 

6  mins 

51  atcs 

1  min 

31  stca 

1  sin 

29  atcs 

0  mim 

41  stca 

2  mins 

34  atcs 

0  mins 

34  smes 

4  mins 

3  atcs 

1  mirv. 

,  15  sees 

1  min 

U  atcs 

0  miim 

32  atcs 

3  mins 

4  atcs 

0  mirs 

58  stca 

4  mins 

50  atcs 

0  mins 

55  stta 

41  mins 

3  atcs 

7  mins 

U  sacs 

atconds 

to  2  hours 

34  mlrutti  15  « 

Source:  unpublished  IPR  materials  by  N.  B.  Peterson, 

et  al.,  PDRI,  1986 
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THE  PROCESS  AND  RESULTS  OF  PREDICTOR  TESTING  IN 
TRADOC'S  SKILLS  SELECTION  AND  SUSTAINMENT  (S3)  PROGRAM 


This  report  summarizes  the  events  and  outcomes  of  the 
testing  component  of  the  Training  and  Doctrine  Command's 
(TRADOC'S)  Skills  Selection  and  Sustainment  (S3)  Program.  A 
short  history  of  the  program  is  followed  by  a  description  of  the 
test  package  and  support  that  the  Army  Research  Institute  (ARI) 
provided  to  the  participating  posts.  Then,  the  testing  operation 
is  described  post  by  post.  Next,  the  outcomes  of  this  program 
are  evaluated  in  terms  of  process  and  impact.  Following  that, 
the  potential  benefits  of  administering  the  new  tests  in  the 
training  base  and  the  pre-enlistment  setting  are  compared. 
Finally,  actual  progress  in  moving  the  tests  toward  use  for 
selecting/  classifying  civilian  applicants  is  described. 


HISTORY 

General  Maxwell  Thurman,  then  the  Commanding  General  (CG)  of 
TRADOC,  initiated  the  S3  program  for  the  purpose  of  improving 
operators'  performance  on  weapons  systems.  His  message  on  11 
December  1987  to  the  CGs  of  Forts  Benning,  Bliss,  Knox,  and  Sill 
(Attachment  1)  directed  them  to  install  new  ARI  tests  of  gunnery 
aptitudes  and  use  scores  on  the  tests  to  select  trainees  for 
gunnery  training  and  positions.  Among  other  things,  he  directed 
the  testing  of  leadership  potential  and  of  spatial  and 
psychomotor  abilities  for  fast  tracking  and  selection  into 
gunnery  specialties  to  start  on  4  January  1988.  GEN  Thurman  also 
directed  that  the  CGs  provide  sustainment  training  for  the  active 
forces,  but  this  paper  does  not  deal  with  the  sustainment 
training  aspect  of  S3. 

ARI  was  tasked  to  provide  software,  hardware,  and  technical 
assistance  to  get  the  testing  going.  The  informal  guidance  to 
ARI  was  to  create  a  test  battery  of  no  more  than  an  hour  that 
would  include  the  Assessment  of  Background  and  Life  Experiences 
(ABLE),  a  test  of  "will  do"  traits,  along  with  the  new  tests  of 
abilities. 

At  the  kickoff  meeting  15  Dec  87  at  Fort  Monroe,  posts 
learned  that  TRADOC  would  not  provide  funds  for  the  program. 

None  of  the  participants,  including  ARI,  was  able  to  meet  the  4 
January  suspense:  Benning,  Knox,  and  Bliss  started  testing  mid— 
to  late  February,  and  Sill  never  started.  ARI  and  the  test  sites 
reported  weekly  to  TRADOC  by  telephone  for  most  of  1988.  Based 
on  the  results  of  the  the  pilot  phase  at  Fort  Bliss,  which  ended 
in  July  1988,  testing  was  not  resumed.  Early  in  1989,  MG 
Spigelmire,  CG,  Fort  Benning,  notified  TRADOC  that  Benning  was 
suspending  testing  for  lack  of  resources.  Benning  resumed  the 
testing  in  the  summer  at  TRADOC'S  direction,  even  though  funding 
was  not  provided. 


ARI's  TESTING  PACKAGE  AND  SUPPORT 

ARI  provided  the  following  materials  and  services  to  the 
field  to  support  the  testing:  over  100  computer-based  test 
stations,  software  for  administering  tests  of  one-  and  two-hand 
tracking,  booklets  for  three  pencil  and  paper  tests  (Mazes, 
Orientation,  and  ABLE) ,  on-site  training  and  computer  tuneup  just 
before  the  start  of  testing,  a  starter  set  of  machine  readable 
answer  sheets,  ongoing  technical  assistance  (mostly  by 
telephone),  and  a  detailed  Test  Administrator's  Manual.  The 
computers  were  Seequa  Chameleons  and  XLs  that  had  had  an  average 
of  300  hours  of  use  previously  in  the  Army's  Project  A.  By  the 
start  of  S3,  Seequa  computers  were  no  longer  manufactured  and  the 
company  with  expertise  to  maintain  them  was  no  longer  in 
business. 

The  Test  Administrator's  Manual  was  supplemented  by  a  Test 
Users'  Decision  Guide  for  making  personnel  decisions  with  the 
test  scores.  In  the  guide,  separate  tables  of  norms  were 
provided  for  ABLE  and  for  a  composite  score  derived  from  the 
spatial  and  psychomotor  tests.  These  norms  were  based  on  large, 
comparable  samples  of  examinees  in  Project  A.  In  the  composite 
score  that  was  formed  from  separate  scores  on  two  spatial  and  two 
psychomotor  tests,  the  psychomotor  component  was  given  a  weight 
double  that  of  spatial,  because  the  psychomotor  tests  have  a 
stronger  relation  with  accuracy  in  gunnery. 

Two  decision  rules  were  suggested  in  the  Decision  Guide, 
both  intended  to  identify  a  larger  pool  of  selectees  than  were 
needed  to  fill  the  relevant  training  slots.  Two  considerations 
led  ARI  researchers  to  build  room  for  local  discretion  into  these 
decision  rules.  First,  we  judged  that  any  decision  rule  that 
removed  local  discretion  completely  would  arouse  resistance  to 
the  S3  program.  Second,  we  believed  that  the  users  in  the  field 
have  unique  information  on  supply  of  and  demand  for  gunners  and 
on  trainees'  performance  during  Basic  Training,  both  of  which 
have  a  legitimate  place  in  influencing  selection  of  gunners. 

One  of  the  proposed  decision  rules  weighted  gunnery 
aptitudes  (i.e.,  scores  on  the  spatial  and  psychomotor  tests)  and 
leadership/motivation  (scores  on  ABLE)  equally;  the  other 
weighted  gunnery  much  more  heavily.  These  decision  rules  were 
offered  to  the  users  as  examples,  to  be  adjusted  in  light  of 
local  realities,  and  not  as  the  last  or  exclusive  word  on 
selecting  gunners. 

sites  returned  filled-in  answer  sheets  and  diskettes  to  ARI, 
where  the  data  are  now  archived,  and  ARI  recycled  fresh  diskettes 
to  the  field.  The  time  for  administering  the  five  tests  turned 
out  to  be  about  75  minutes.  For  hand  scoring  the  three  pencil- 
and-paper  tests,  it  took  two  persons  about  two  hours  to  do  50 
examinees.  Late  in  July  1988,  ARI  converted  ABLE  from  pencil- 


and-paper  to  computerized  form,  which  cut  the  labor  of  hand 
scoring  by  more  than  one  half.  In  all,  it  took  major  commitments 
of  time  by  10  ARI  Headquarters  and  Field  Unit  researchers  plus 
contractor  support  to  prepare  and  install  the  testing  package. 


THE  PROCESS  AT  THE  TRAINING  POSTS 

At  the  mutual  request  of  ARI  and  the  participating  posts, 
TRADOC  agreed  for  the  testing  to  undergo  a  shakedown  and 
evaluation  period  before  the  scores  would  be  used  for  making 
personnel  decisions.  The  three  posts  that  conducted  the  testing 
made  use  of  that  grace  period. 

At  the  start  of  each  testing  session,  examinees  were  told 
the  following:  the  test  scores  might  be  used  to  pick  a  specialty 
or  type  of  training  for  them;  the  scores  would  go  into  their 
personnel  records;  and,  therefore,  it  was  in  their  best  interests 
to  do  the  best  that  they  could  on  the  tests.  In  other  words,  the 
examinees  were  told  that  these  tests  counted. 

Fort  Benninq 

The  staff  of  the  Reception  Battalion  administered  the  tests, 
scored  them,  transmitted  the  scores  to  Headquarters,  U.  A.  Army 
Infantry  Training  Center  (ITC) ,  and  mailed  the  completed  answer 
sheets  and  filled  diskettes  back  to  ARI.  Insofar  as  possible, 
all  Regular  Army  IIX  receptees  were  tested.  Toward  the  end  of 
the  AIT  phase  of  training,  the  29th  Infantry  Regiment  conducted 
qualification  testing  on  all  IIH  students.  That  testing  covered, 
among  other  things,  gunnery  performance  on  a  high  fidelity  TOW 
simulator  at  a  gunnery  range. 

The  results  of  S3  testing  were  continuously  monitored  by  the 
Analysis  and  Studies  Office  of  the  Directorate  of  Training  and 
Doctrine  (DOTD)  in  the  Infantry  School.  Researchers  in  that 
office  gathered  the  spatial/psychomotor  test  scores  from  the 
Reception  Battalion  and,  up  to  spring  1989,  drove  out  to  the 
gunnery  range  periodically  to  gather  the  data  from  the  gunnery 
exercises.  Their  analyses  and  briefing  in  June  1988  led  the 
Commander  of  the  Infantry  Training  Center  to  approve  use  of  the 
spatial  and  psychomotor  test  scores  for  selecting  IIX  Basic 
Trainees  into  IIH.  Although  it  was  not  approved  for  use  in 
making  personnel  decisions,  ABLE  continued  to  be  administered  in 
the  S3  package.  The  Analysis  and  Studies  Office  also  designed 
forms  for  recording  IIH  trainees'  scores  on  selected  hands-on 
tasks  (e.g.,  prepare  range  card;  assemble  weapon),  worked  with 
the  29th  Infantry  Regiment  to  set  up  a  procedure  for  filling  out 
the  forms,  gathered  the  completed  forms,  and  mailed  them  to  ARI. 
By  the  end  of  the  S3  Program,  the  new  tests  had  been  taken  by 
about  18,000  receptees  at  Fort  Banning.  The  ARI  Field  Unit  at 
Fort  Banning  supported  the  S3  program  throughout  by  keeping  the 
participants  at  ARI  Headquarters  and  Fort  Banning  informed  of 
perceptions  and  realities  at  both  ends  of  the  operation. 


Fort  Knox 


At  Knox,  the  new  tests  were  housed  at  the  Reception 
Battalion  and  run  almost  to  the  program's  end  by  the  SQT  Office 
of  the  Armor  School.  Scores  on  the  S3  tests  were  transmitted  to 
the  Armor  Training  Brigade,  the  Office  of  the  Chief  of  Armor,  the 
ARI  Field  Unit  on  posts,  and,  by  mail,  to  ARI  Headquarters.^ 
Examinees '  gunnery  scores  on  the  Institutional  Conduct  of  Fire 
Trainer  (ICOFT)  later  in  training  were  gathered  and  evaluated  by 
the  local  ARI  Field  Unit.  Their  analyses  led  the  Fort  Knox  CG, 

MG  Tait,  to  agree  in  September  1988  to  permit  scores  on  the 
spatial  and  psychomotor  tests  to  be  considered,  as  one  of  several 
types  of  evidence,  in  selecting  volunteers  into  the  Excellence  in 
Armor  Program.  ABLE  continued  to  be  administered  along  with  the 
ability  tests.  By  the  end  of  the  S3  Program,  about  7,500 
receptees  had  taken  the  new  tests  there. 


Fort  Bliss 

Here,  the  ARI  Field  Unit  conducted  the  predictor  testing  on 
post  and  the  performance  testing  at  White  Sands  Missile  Range. 

At  the  direction  of  the  Air  Defense  Artillery  School,  the  test 
package  included  only  the  psychomotor  and  spatial  tests.  The 
examinees  were  students  in  16S  and  16P  who  were  close  to 
finishing  the  AIT  phase  of  their  OSUT.  On  one  day  they  took  the 
S3  tests,  and  within  the  next  two  days  were  tested  on  high 
fidelity  simulators  of  their  respective  weapons  systems.  A  team 
of  researchers  at  ARI's  local  Field  Unit  and  Headquarters 
g'\^a.luated  and  reported  the  data.  Based  on  an  absence  of  positive 
results  at  Fort  Bliss,  the  testing  was  not  continued  beyond  the 
pilot  phase.  Only  101  cases  were  tested  there  owing  to  the  very 
limited  funding  for  gathering  performance  data  on  the  Realistic 
Air  Defense  Engagement  System  (RADES) . 


Fort  Sill 

ARI  held  meetings  with  the  Field  Artillery  Center  and  School 
at  Fort  Sill  in  August  and  November  1988,  and  again  in  February 
1989  when  the  test  package  was  installed.  Late  in  1988,  the 
Field  Artillery  Center  and  School  decided  to  include  four  MOS  in 
its  S3  testing,  each  with  a  unique  set  of  spatial  and  psychomotor 
tests.  ARI  designed  an  expanded  test  package  and  installed  it  at 
Fort  Sill's  Reception  Battalion  in  February  1989.  It  included 
five  pencil  and  paper  tests  of  spatial  abilities  and  software  to 
administer  as  many  as  seven  computerized  tests,  the  exact  choice 
of  tests  varying  with  MOS.  By  the  close  of  the  S3  Program, 
repairs  had  not  been  completed  on  the  building  that  would  have 
housed  the  testing  operation,  and  so  testing  never  got  under  way. 


DESCRIPTION  AND  EVALUATION  OF  THE  OUTCOMES 
Test  Administration 

The  team  of  field  and  ARI  personnel  succeeded  in  setting 
up,  starting,  and  maintaining  a  successful  testing  operation  for 
eighteen  months.  Over  25,000  entry-level  students  took  the  new 
S3  tests.  Given  that  the  program  had  been  launched  suddenly, 
at  the  participants'  expense,  and  with  aging  hardware,  the  posts' 
effective  conduct  of  the  testing  program  was  a  considerable 
achievement.  The  effectiveness  of  the  testing  is  proven  both  by 
the  numbers  tested  and  by  thie  findings  that  the  new  tests  work 
well  for  identifying  students  with  aptitudes  to  be  good  tank  and 
anti-tank  gunners,  as  discussed  next. 

Test  Validities 

In  the  S3  Program,  the  high  validities  of  spatial  and 
psychomotor  tests  which  had  been  found  in  1986—7  with  ARI  doing 
the  testing,  were  replicated;  the  tests'  power  to  predict  levels 
of  performance  on  simulators  of  tank  and  anti-tank  gunnery  held 
up  in  the  S3  setting  (Smith  &  Walker,  1988;  Graham,  ^89).  In 
MOS  IIH,  students  who  scored  in  the  upper  one-third  in  S3  gunnery 
aptitudes  (i.e.,  on  the  norms  of  spatial  and  psychomotor  tests) 
attained  Expert  or  First  Class  levels  of  accuracy  on  the  TOW 
simulator  at  a  rate  12%  higher  than  the  prevailing  average. 
Seventeen  percent  more  of  those  in  the  top  third  attained  the  top 
two  levels  of  gunnery  qualification  than  those  in  the  bottom  two- 
thirds.  A  small  (N  =  59  students)  live  fire  exercise  at  Fort 
Banning  in  June  1988  confirmed  those  findings,  with  11%  more  of 
the  upper  third  (in  spatial/psychomotor  abilities)  hitting  the 
moving  target  2,000  yards  down  range  on  their  single  shot  than 
those  in  the  lower  two-thirds.  Although  that  result  was  not 
statistically  significant,  in  the  context  of  the  other  TOW  data, 
it  is  entirely  credible.  Students  in  the  upper  third  of  S3 
gunnery  aptitudes  also  trained  up  to  the  qualifying  level  faster: 
41%  of  them  qualified  on  the  earliest  possible  gunnery  table, 
while  only  12%  of  the  others  did.  The  exact  figures  varied  a  bit 
as  time  passed,  but  the  advantage  of  the  high  spatial/psycho¬ 
motor  students  endured. 

An  additional  benefit  was  gained  from  the  performance  data 
that  the  Studies  and  Analysis  Office  at  Fort  Banning  collected  at 
the  firing  range:  ARI's  analyses  of  these  data  for  TRADOC's 
DCSRM  have  been  included  in  the  report  on  the  Soldier  Performance 
Research  Project  (Block,  Polich,  &  Horne,  in  press)  to  confirm 
that  quality,  as  defined  by  scores  on  ASVAB,  has  a  positive 
impact  on  both  firing  accuracy  and  hands-on  tasks  on  the  TOW. 

The  results  of  S3  at  Fort  Knox  that  are  reported  here  are 
from  the  work  of  Dr.  Scott  Graham  of  the  ARI  Field  Unit  there 


(Graham,  1988,  1989).  In  19K,  students  who  scored  in  the  upper 
third  in  S3  Gunnery  aptitudes  had  a  hit  rate  7%  higher  on  the 
ICOFT  than  the  prevailing  norm,  and  13%  higher  than  the  lower ^ 
two-thirds.  That  finding  exactly  replicates  the  differences  in  a 
sample  of  officers  in  earlier  research  (Smith  &  Graham,  1987) . 

To  equal  the  hits  of  these  high  ability  students,  11%  more  tanks 
manned  by  average  gunners  would  be  needed. 

In  comparison  with  students  in  the  Excellence  in  Armor  (EIA) 
Program,  who  averaged  71%  hits  on  the  20th  hour  of  ICOFT, 

19KS  in  the  upper  third  of  S3  gunnery  aptitudes  averaged  70%. 

Even  on  the  input  side,  the  EIA  selection  system  is  capturing 
some  of  the  same  information  as  the  new  tests:  EIAs  scored 
significantly  higher  on  the  spatial  and  psychomotor  tests  than 
the  non-EIAs  (Graham,  1989,  p.  14).  However,  there  is  still  a 
strong  correlation  of  S3  test  scores  with  the  ICOFT  scores  of  the 
EIA  students.  That  finding  means  that  scores  on  tests  of  spatial 
and  psychomotor  abilities  provide  sufficient  unique  information 
to  aid  in  selecting  EIAs  so  as  to  improve  their  gunnery 
performance. 

On  ABLE  as  well,  EIAs  score  significantly  higher  than  non- 
EIAs  who  were  tested  in  S3.  The  non-EIAs'  average  ABLE  score 
fell  at  the  54th  percentile  in  the  norms,  while  the  EIAs'  fell  at 
the  68th  percentile.  Thus,  the  EIA  selection  process  operates  as 
though  it  were  selecting  from  the  upper  one-third  on  ABLE. 

At  Fort  Bliss,  scores  on  the  new  tests  were  not  related  to 
trainees'  performance  in  gunnery  on  the  Realistic  Air  Defense 
Engagement  System  (RADES) .  ARI  and  the  Air  Defense  Artillery 
School  agreed  that  the  two  air  defense  weapons  systems  in  the  S3 
testing.  Stinger  and  Chaparral,  probably  do  not  require  the 
abilities  measured  by  new  ARI  tests  nearly  as  much  as  they 
require  other  abilities  (e.g.,  vision). 


Credibility  of  the  Results 

Based  on  data  from  the  first  few  months  of  testing, 
leadership  at  Banning  and  Knox  agreed  to  permit  the  test  scores 
to  enter  into  the  decision  process  for  selecting  the  relevant 
categories  of  students.  Thus  the  tests  and  the  evidence  of  their 
effectiveness  were  credible  to  users  in  the  field.  The  negative 
results  at  Fort  Bliss  were  credible  as  well,  both  to  the  field 
and  to  ARI.  So  the  test  results  provided  good  information  for 
evaluating  the  utility  of  the  test  package  at  all  three  posts. 

Because  the  S3  tests  have  repeatedly  shown  such  high 
potential  benefit  to  the  Army  for  tank  and  anti-tank  gunnery,  ARI 
has  briefed  the  results  of  the  program  to  a  variety  of  senior 
decision  makers  in  the  Army  and  DoD.  These  include  the  following 
General  Officers  and  Senior  Executive  Service  officials:  CG  SID 
(MG  Waller,  Jul  88) ,  CG  V  Corps  (LTG  Woodmansee,  Jul  88) ,  CG  3AD 
(MG  Joulwan,  Jul  88),  CG  Fort  Knox  (MG  Tait,  Sep  88),  DCSPER  of 


the  Army  (LTG  Ono,  Sep  88),  VCSA  (GEN  Brown,  Sep  88),  DoD's 
Director  of  Military  Accession  Policy  (Dr.  Sellman,  Dec  88),  the 
Assistant  (to  DoD's  Director  of  Environmental  and  Life  Sciences) 
for  Training  and  Personnel  Systems  Technology  (Dr.  Alluisi,  12/88 
and  6/89),  CG  Fort  Banning  (MG  Spigelmire,  Feb  89),  DoD's 
Assistant  Military  Deputy  for  Research,  Development,  and 
Acquisition  (MG  Beltson,  Jul  89),  ODCSPER's  Deputy  for  Military 
Personnel  Management  (BG  Stroup,  Aug  89) ,  and  the  ADCSPER  of  the 
Army  (MG  Budge,  Sep  89) . 


Impact  on  Selecting  Gunners 

One  major  aspect  of  the  S3  testing  operation  was  not  a 
success :  the  test  scores  did  not  have  an  impact  on  actual 

personnel  selection.  Four  components  of  a  system  for  effective 
use  of  the  scores  were  not  uniformly  installed.  First,  at  Fort 
Knox,  the  post-level  information  management  systems  for  getting 
the  test  scores  to  decision  makers  were  not  set  up  and 
maintained.  Test  scores  did  get  out  of  the  Reception  Battalions 
to  the  Training  Brigade  at  Knox,  but  not  out  to  the  students' 
units.  As  a  result,  decision  makers  on-site  were  not  able  to  use 
the  test  scores  for  selecting  trainees  based  on  aptitudes  for 
gunnery. 

Second,  a  system  was  not  set  up  or  maintained  for  getting 
the  test  scores  or  some  sign  of  them,  like  an  Additional  Skill 
Indicator,  into  students*  permanent  personnel  records.  Thus,  the 
students'  future  gaining  units  had  no  way  of  identifying 
individuals  with  high  spatial  and  psychomotor  abilities  so  as  to 
make  best  use  of  their  talents.  In  the  case  of  the  EIA  Program, 
that  missing  link  of  information  is  less  of  a  loss  because  fast 
tracking  moves  the  EIAs  ahead  in  the  progression  toward  gunners' 
seats.  But  for  llHs,  the  information  on  gunners'  aptitudes  now 
exists  in  an  easy-to-retrieve  form  only  in  ARI's  databases. 

Third,  even  where  the  test  scores  had  gotten  into  the  hands 
of  those  who  select  gunners  (viz.,  at  Fort  Benning) ,  procedures 
for  using  the  information  effectively  to  make  the  relevant 
personnel  decisions  were  not  installed.  Just  getting  the  scores 
into  the  users'  hands  does  not  guarantee  that  the  scores  are 
usable  or  used;  practical,  believable  procedures  need  to  be 
available  for  decision  makers  to  employ  in  consuming  the  test 
scores.  At  Fort  Benning,  even  though  the  test  scores  did  get  to 
students'  units,  other  factors  drove  assignment  of  llXs  to 
specific  Infantry  MOS. 

Fourth  and  equally  important,  the  marketing,  training,  and 
monitoring  necessary  to  institutionalize  use  of  the  test  scores 
did  not  take  place.  Of  these  four  components,  probably  only  the 
first  could  have  been  effected  in  a  relatively  short  time;  but 
even  that  component  would  have  required  additional  resources  that 
were  not  available  (e.g.,  networking  hardware  and  software  for 
the  computers ;  machines  for  scanning  the  answer  sheets  at  the 


test  sites) .  In  summary,  S3  was  successful  in  generating  useful 
scores  on  new  tests,  but  it  did  not  follow  through  with  the 
institutional  changes  needed  for  getting  the  test  scores  used. 
Bringing  about  the  changes  would  have  required  continuing  active 
involvement  together  by  TRADOC,  ARI,  and  each  post. 


POTENTIAL  UTILITY  OF  THE  S3  TESTS  IN  THE  TRAINING  BASE 

Now  that  a  strong  relationship  between  spatial/psychomotor 
abilities  and  gunnery  perforaance  has  been  replicated  in  the  S3 
setting,  the  question  remains,  "How  could  the  benefit  of  the 
tests  be  realized?"  In  this  section,  the  potential  benefit  of 
administering  the  tests  at  Forts  Knox  and  Banning  for  selecting 
gunners  is  discussed. 


Fort  Knox 


Even  though  it  is  designed  as  a  leadership,  not  a  gunnery, 
program.  Excellence  In  Armor  (EIA)  appears  to  be  doing  an^ 
effective  job  of  selecting  trainees  who  have  gunners'  aptitudes 
without  using  scores  on  spatial  or  psychomotor  tests.  Still,  the 
finding  that  EIA  students'  scores  on  the  S3  psychomotor  and 
spatial  battery  correlate  significantly  with  their  performance  on 
ICO FT  shows  that  the  new  tests  could  aid  in  selection  into  EIA. 
Whether  the  EIA  program  can  continue  to  be  as  successful  in 
producing  superior  gunners  as  the  quality  of  accessions  varies 
over  time  is  unknown. 

EIA  selects  students  based  on  a  broad  set  of  indicators 
(ASVAB  scores,  academic  performance,  fitness,  leadership,  etc.) 
that  accumulate  by  the  end  of  the  first  seven  weeks  of  training. 
It  is  not  surprising  that  this  "seven  week  test"  provides  good 
information  on  students'  leadership  potential.  Also,  the  "seven 
week  test"  has  the  advantages  of  making  use  of  ASVAB  scores  and 
direct  observation  (i.e. ,  two  persuasive  types  of  evidence) ,  it 
is  a  local  product,  it  is  in  place,  and  it  appears  to  enjoy 
strong  support  on  post. 

The  choice  between  using  ABLE  scores  and  the  "seven  week 
test"  is  a  false  choice:  ABLE  scores  could  be  added  into  the  set 
of  evidence  that  is  examined  in  selecting  volunteers  ^  for  fast 
tracking.  When  ABLE  scores  from  pre-enlistment  testing  (see 
below)  become  available  in  receptees '  personnel  records,  then 
they  could  be  used  as  easily  as  ASVAB  scores  are  now  in  selecting 
volunteers  into  EIA. 

Although  there  is  considerable  evidence  that  the  S3  tests 
have  good  information  for  use  in  selecting  EIAs,  the  question 
remains  as  to  the  added  utility  of  scores  from  the  S3  tests.  It 
is  not  obvious  that  the  S3  tests  add  enough  new  information  to 
warrant  the  cost  of  the  testing  program  at  Fort  Knox.  The  EIA 
program  is  different  from  settings  where  gunners  enter  their  duty 


positions  essentially  unselected  for  special  gunnery  aptitudes. 

One  relevant  research  question  for  TRADOC  and  ARI  is  how 
widely  and  systematically  scores  on  Conduct  of  Fire  Trainers  are 
used  by  TO&E  units  to  select  and  de-select  gunners.  If  units  use 
COFT  scores  for  putting  and  keeping  individuals  in  gunners* 
seats,  then  they  are  selecting  on  the  basis  of  special  gunners' 
aptitudes . 

Fort  Bennina 

At  Fort  Benning,  the  present  system  for  classifying  llXs 
into  the  four  Infantry  MOS  does  not  have  the  effect  of  taking 
spatial  and  psychomotor  abilities  into  account.  As  a  result, 
students  with  the  full  range  of  spatial  and  psychomotor  abilities 
enter  IIH,  and  their  average  performance  on  the  M-70  TOW 
simulator  is  markedly  lower  than  it  could  be.  Using  the  new 
ability  tests  as  the  primary  selector  of  llXs  into  IIH  would 
significantly  improve  TOW  gunnery  performance. 

But  even  if  the  administrative  ground  work  and  support  for 
using  the  test  scores  were  in  place,  three  aspects  of  personnel 
processing  at  Benning  would  have  to  be  dealt  with  before  the  new 
ability  tests  could  have  a  significant  effect  on  assignment  to 
MOS.  First,  the  distribution  of  high  quality  students  (in  terms 
of  the  ASVAB  General  Technical  score)  across  MOS  is  constrained 
by  a  locally  developed  system  for  spreading  quality  across  MOS. 
Second,  some  students  enter  OSUT  at  Fort  Benning  with  enlistment 
contracts  that  make  it  necessary  to  put  them  in  MOS  IIH.  There 
is  no  paperwork  mechanism  now  that  allows  post— enlistment  tests 
to  change  those  contracts  for  students  with  low  spatial  and 
psychomotor  abilities.  Finally,  occasional  surges  in  demand  for 
TOW  gunners  make  it  necessary  to  channel  students  into  IIH 
irrespective  of  their  aptitudes  for  gunnery. 

There  may  be  ways  to  incorporate  scores  on  spatial  and 
psychomotor  tests  into  the  local  system  for  distributing  quality 
soldiers  and  the  correlation  of  ASVAB  scores  with  the  gunner's 
aptitudes  is  low  enough  that  IIH  could  be  filled  without  IIH 
having  to  monopolize  the  high-ASVAB  students.  But  the  other  two 
constraints,  enlistment  contracts  and  surges  in  demand,  seem 
inescapable.  So  the  possible  impact  of  the  new  tests  at  Fort 
Benning  is  limited  to  some  unknown  degree  by  the  realities  of 
personnel  processing  for  llHs. 


MAXIMIZING  THE  UTILITY  OF  THE  S3  TESTS 

A  solution  to  these  limits  on  the  impact  of  the  new  tests 
at  Forts  Benning  and  Knox  has  occurred  to  personnel  at  both  the 
training  schools  and  ARIs  install  the  tests  in  pre— enlistment 
processing  for  selecting  and  classifying  civilian  applicants.  By 
siting  the  tests  in  the  pre-enlistment  setting,  the  Army  would  be 
able  to  identify  persons  with  gunners'  aptitudes  from  the  pool  of 


all  applicants,  and  would  be  able  to  assign  qualified  individuals 
to  all  of  the  MOS  that  require  spatial  and  psychomotor  abilities. 
Since  about  400,000  Army  applicants  take  ASVAB  annually,  and 
accessions  average  about  115,000  annually,  the  number  of  high 
spatial  and  psychomotor  individuals  who  could  be  assigned  to 
gunnery  specialties  would  be  vastly  larger  than  the  numbers  now 
entering  IIX  and  19K.  Perhaps  just  as  important,  the  numbers  of 
persons  with  low  aptitudes  for  gunnery  who  could  be  steered  away 
from  gunnery  occupations  would  be  enormous.  Such  a  system  could 
raise  the  level  of  gunners'  aptitudes  in  the  llHs  and  19Ks  by 
doing  the  person- job  match  before  training,  which  was  the 
original  intent  of  S3. 

The  benefits  to  the  Army  would  be  improved  trainability  and 
performance  in  critical  skills  for  relevant  MOS.  These  benefits 
to  TRADOC  and  the  active  forces  would  be  attained,  however,  at 
some  cost  to  MEPCOM.  For  this  reason,  implementation  of  such  new 
ability  tests  at  MEPCOM  sites  will  not  occur  until  the  cost- 
effectiveness  of  installing  a  computerized  testing  system  has 
been  evaluated  and  found  positive. 

At  this  writing,  the  Department  of  Defense  is  preparing 
such  an  evaluation.  In  1990-91,  the  DoD  Enhanced  Computer 
Assisted  Testing  (ECAT)  Project  will  be  conducted  to  evaluate  a 
battery  of  Army  and  Navy  computerized  tests,  including  the  four 
S3  ability  tests.  For  that  project,  several  pencil-and-paper 
tests  of  spatial  abilities  will  be  converted  to  the  computerized 
medium.  Data  collection  will  be  carried  out  in  the  training 
bases  of  all  of  the  Services,  but  hardware,  software,  and 
staffing  will  all  be  supplied  to  sites  via  an  existing  Navy 
contract.  Upon  receipt  of  details  from  DoD,  ODCSPER  will  open 
formal  coordination  with  TRADOC  on  this  new  project. 

Overall,  the  objective  of  ECAT  is  to  see  whether  it  would  be 
cost-effective  to  computerize  all  pre-enlistment  testing.  The 
new  tests  are  to  be  evaluated  as  supplements  to  ASVAB,  not  as 
replacements.  As  a  final  note  of  evaluation  on  S3,  results  from 
the  S3  program  have  influenced  DoD's  choice  of  tests  for  ECAT;  so 
although  the  S3  testing  is  ended,  its  impact  continues  in  ECAT's 
pre-enlistment  tests  of  the  future.  The  potential  long  run 
benefit  of  those  tests  -  across  all  relevant  specialties  in  all 
Services  -  is  very  great. 

As  for  testing  the  will-do  attributes  which  ABLE  measures, 
administering  ABLE  in  the  pre-enlistment  setting  would  have  the 
same  advantages  of  reaching  the  full  applicant  pool  and  helping 
assignment  to  all  MOS.  DoD  has  included  a  form  of  ABLE  in  its 
Applicant  Screening  Profile  (ASP) ,  which  has  already  been  printed 
by  MEPCOM.  At  present,  ASP  awaits  final  approval  for  use  in 
testing  applicants  to  all  of  the  Services,  and  so  is  on  the  verge 
of  implementation. 

The  ASP  is  intended  primarily  as  a  selection  screen  to 
reduce  attrition,  but  economists  in  ARI's  Manpower  and  Personnel 


Policy  Research  Group  have  developed  strategies  for  using  ABLE 
for  classification  (i.e.,  person- job  match)  as  well.  Once  the 
amount  of  attrition  is  reduced  by  screening  out  the  highest  risk 
applicants,  the  cost  of  attrition  can  be  reduced  further  by 
distributing  the  highest  risk  accessions  across  MOS  where  the 
cost  of  training  is  not  high. 
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Introduction 


This  codebook  is  being  provided  to  the  Defense  Training  end  Performance 
Data  Center  (TPDC)  in  conjunction  with  the  transmission  of  data  tapes  to  TPDC. 
These  tapes  contain  data  from  the  Army’s  Project  A,  a  large-scale  multiyear 
research  effort  designed  to  improve  the  selection,  classification  and 
utilization  of  Army  Enlisted  Personnel.  The  Initial  large  data  collection 
effort  of  Project  A  was  the  Concurrent  Validation  (CV)  in  which  predictor  and 
performance  data  were  collected  concurrently  from  soldiers  after  12  to  27 
months  of  service.  The  CV  resulted  in  the  collection  of  more  than  fifteen 
million  data  points  from  approximately  9000  soldiers  in  19  Military 
Occup&ti.oii&l  SpcciEltics  (MOS)* 

In  order  to  effectively  use  the  large  amount  of  data  collected  in  the 
field  during  the  CV,  as  well  as  variables  obtained  from  other  sources,  a 
Longitudinal  Research  Data  Base  (LRDB)  was  established.  Due  to  the 
requirements  of  various  types  of  data  analyses  to  be  performed,  it  was 
critical  that  the  files  in  the  LRDB  be  efficiently  designed  and  well 
documented.  Generally,  item  and  step  level  data  are  maintained  on  tapes  and 
^  ^rged  with  macro  level  variable  files  as  required.  The  large  separate  MOS 
files  for  the  CV  contain  the  primary  macro  level  variables  used  for  analysis 
Including  Task  Scores,  Scale  Scores,  Construct  Scores,  Mean  Ratings  Scores, 
Imputation  Flags  and  Residual  Scores  as  well  as  demographic,  accession  and 
Anny<-collected  variables. 

This  paper  presents  one  of  the  Concurrent  Validation  Codebooks  developed 
by  the  LRDB  manager  and  her  team  for  the  nine  MOS  for  which  Job  Knowledge 
Tests,  Hands-On  Tests  and  MOS-specific  Ratings  were  developed. 


